{"id":250931,"date":"2017-02-06T05:59:38","date_gmt":"2017-02-06T12:59:38","guid":{"rendered":"http:\/\/css-tricks.com\/?p=250931"},"modified":"2017-02-06T05:59:38","modified_gmt":"2017-02-06T12:59:38","slug":"really-makes-static-site-generator","status":"publish","type":"post","link":"https:\/\/css-tricks.com\/really-makes-static-site-generator\/","title":{"rendered":"What Really Makes a Static Site Generator?"},"content":{"rendered":"
I talk a lot about static site generators, but always about using<\/em> static site generators. In most cases, it may seem like a black box. I create a template and some Markdown and out comes a fully formed HTML page. Magic!<\/p>\n But what exactly is<\/em> a static site generator? What goes on inside that black box? What kind of voodoo is this?<\/p>\n In this post, I want to explore all of the parts that make up a static site generator. First, we’ll discuss these in a general fashion, but then we’ll take a closer look at some actual code by delving deep inside HarpJS<\/a>. So, put your adventurer’s cap on and let’s start exploring.<\/p>\n <\/p>\n Why Harp?<\/strong> For two reasons. The first is that HarpJS is, by design, a very simple static site generator. It doesn’t have a lot of the features that might cause us to get lost exploring a more comprehensively full-featured static site generator (like Jekyll<\/a> for instance). The second, much more practical, reason is that I know JavaScript and don’t know Ruby very well.<\/p>\n The truth is, a static site generator is a pretty simple concept. The key ingredients to a static site generator are typically:<\/p>\n That’s it. If you’re thinking, “Hey… I could build that!” you are probably correct. Things start to get complicated though when you start to expand the functionality, as most static site generators do.<\/p>\n So, let’s look at how Harp handles this.<\/p>\n Let’s look at the basics of how Harp handles the key ingredients described above. Harp offers more than this handful of functionality, but, for the sake of our examination, we’ll stick to those items.<\/p>\n First, let’s discuss the basics of Harp.<\/p>\n Harp supports Jade<\/a> and EJS<\/a> (for templating) and Markdown as its lightweight markup language (for content). Note that while Jade is now called Pug, Harp has not officially transitioned in their documentation or code, so we’ll stick with Jade here. Harp also offers support for other preprocessing such as Less, Sass, and Stylus for CSS and CoffeeScript for JavaScript.<\/p>\n By default Harp does not require much in the way of configuration or metadata. It tends to favor convention over configuration<\/a>. However, it allows for specific metadata and configuration using JSON. It differs from many other static site generators in that file metadata is contained outside of the actual file within a `_data.json` file.<\/p>\n While it is configurable to a degree, Harp has certain established guidelines for how to structure files. For example, in a typical application, the files that are served fall within a Lastly, Harp offers a basic local web server for testing that includes some configurable options. And, of course, it will compile the finished HTML, CSS and JavaScript files for deployment.<\/p>\n Since much of what makes a static site generator are rules and conventions, the code centers around the actual serving and compiling (for the most part). Let’s dig in.<\/p>\n In Harp, serving your project is usually done by executing While the function looks simple, obviously there is a ton going on within middleware<\/a> that isn’t illustrated here.<\/p>\n The rest of this function opens up a server with the options you specify (if any). Those options include a port, an IP to bind to and a directory. By default the port is 9000 (not 9966 as you might guess by the code), the directory is the current one (i.e. the one Harp is running in) and the IP is Staying within index.js<\/a>, let’s take a look at the The first portion defines the output path as specified by the call to The next part starts by calling the You may also notice a call to something called The next portion of code, as it states, tries to prevent you from specifying an output directory that would inadvertently overwrite your source code (which would be bad as you’d lose any work since your last commit).<\/p>\n The As I discussed, Terraform does the grunt work for compiling the Jade, Markdown, Sass and CoffeeScript into HTML, CSS and JavaScript (and assembling these pieces as defined by Harp). Terraform is made up of a number of files that define its processors for JavaScript, CSS\/stylesheets, and templates (which, in this case, includes Markdown).<\/p>\nThe Basics of a Static Site Generator<\/h3>\n
\n
Getting to the Harp of the Matter<\/h3>\n
Harp Basics<\/h3>\n
public<\/code> directory. Also, any file or folder prefaced by an underscore will not be served.<\/p>\n
Let’s Look at Harp’s Actual Source Code<\/h4>\n
The Server Function<\/h4>\n
harp server<\/code> from the command line. Let’s look at the code for that function<\/a>:<\/p>\n
exports.server = function(dirPath, options, callback){\r\n var app = connect()\r\n app.use(middleware.regProjectFinder(dirPath))\r\n app.use(middleware.setup)\r\n app.use(middleware.basicAuth)\r\n app.use(middleware.underscore)\r\n app.use(middleware.mwl)\r\n app.use(middleware.static)\r\n app.use(middleware.poly)\r\n app.use(middleware.process)\r\n app.use(middleware.fallback)\r\n\r\n return app.listen(options.port || 9966, options.ip, function(){\r\n app.projectPath = dirPath\r\n callback.apply(app, arguments)\r\n })\r\n}<\/code><\/pre>\n
0.0.0.0<\/code>.<\/p>\n
The Compiler Function<\/h4>\n
compile<\/code> function next.<\/p>\n
exports.compile = function(projectPath, outputPath, callback){\r\n\r\n \/**\r\n * Both projectPath and outputPath are optional\r\n *\/\r\n\r\n if(!callback){\r\n callback = outputPath\r\n outputPath = \"www\"\r\n }\r\n\r\n if(!outputPath){\r\n outputPath = \"www\"\r\n }\r\n\r\n\r\n \/**\r\n * Setup all the paths and collect all the data\r\n *\/\r\n\r\n try{\r\n outputPath = path.resolve(projectPath, outputPath)\r\n var setup = helpers.setup(projectPath, \"production\")\r\n var terra = terraform.root(setup.publicPath, setup.config.globals)\r\n }catch(err){\r\n return callback(err)\r\n }\r\n\r\n\r\n \/**\r\n * Protect the user (as much as possible) from compiling up the tree\r\n * resulting in the project deleting its own source code.\r\n *\/\r\n\r\n if(!helpers.willAllow(projectPath, outputPath)){\r\n return callback({\r\n type: \"Invalid Output Path\",\r\n message: \"Output path cannot be greater then one level up from project path and must be in directory starting with `_` (underscore).\",\r\n projectPath: projectPath,\r\n outputPath: outputPath\r\n })\r\n }\r\n\r\n\r\n \/**\r\n * Compile and save file\r\n *\/\r\n\r\n var compileFile = function(file, done){\r\n process.nextTick(function () {\r\n terra.render(file, function(error, body){\r\n if(error){\r\n done(error)\r\n }else{\r\n if(body){\r\n var dest = path.resolve(outputPath, terraform.helpers.outputPath(file))\r\n fs.mkdirp(path.dirname(dest), function(err){\r\n fs.writeFile(dest, body, done)\r\n })\r\n }else{\r\n done()\r\n }\r\n }\r\n })\r\n })\r\n }\r\n\r\n \/**\r\n * Copy File\r\n *\r\n * TODO: reference ignore extensions from a terraform helper.\r\n *\/\r\n var copyFile = function(file, done){\r\n var ext = path.extname(file)\r\n if(!terraform.helpers.shouldIgnore(file) && [\".jade\", \".ejs\", \".md\", \".styl\", \".less\", \".scss\", \".sass\", \".coffee\"].indexOf(ext) === -1){\r\n var localPath = path.resolve(outputPath, file)\r\n fs.mkdirp(path.dirname(localPath), function(err){\r\n fs.copy(path.resolve(setup.publicPath, file), localPath, done)\r\n })\r\n }else{\r\n done()\r\n }\r\n }\r\n\r\n \/**\r\n * Scan dir, Compile Less and Jade, Copy the others\r\n *\/\r\n\r\n helpers.prime(outputPath, { ignore: projectPath }, function(err){\r\n if(err) console.log(err)\r\n\r\n helpers.ls(setup.publicPath, function(err, results){\r\n async.each(results, compileFile, function(err){\r\n if(err){\r\n callback(err)\r\n }else{\r\n async.each(results, copyFile, function(err){\r\n setup.config['harp_version'] = pkg.version\r\n delete setup.config.globals\r\n callback(null, setup.config)\r\n })\r\n }\r\n })\r\n })\r\n })\r\n\r\n}<\/code><\/pre>\n
harp compile<\/code> via the command line (source here<\/a>). The default, as you can see, is
www<\/code>. The callback is a callback function passed by the command line utility which is not configurable.<\/p>\n
setup<\/code> function in the helpers module<\/a>. For the sake of brevity, we won’t go into the specific code of the function (feel free to look for yourself), but essentially it reads the site configuration (i.e.
harp.json<\/code>).<\/p>\n
terraform<\/code>. This will come up again within this function. Terraform<\/a> is actually a separate project required by Harp that is the basis of its asset pipeline<\/a>. The asset pipeline is where the hard work of compiling and building the finished site gets done (we’ll look at Terraform code in a little bit).<\/p>\n
compileFile<\/code> and
copyFile<\/code> functions are fairly self-explanatory. The
compileFile<\/code> function relies on Terraform to do the actual compilation. Both of these functions drive the
prime<\/code> function which uses a helper function (
fs<\/code>) to walk the directories, compiling or copying files as necessary in the process.<\/p>\n
Terraform<\/h4>\n