Creating A Markdown Blog

Creating A Markdown Blog

One thing that I constantly tell my students is that if you can build a blog, you can build pretty much anything. There’s nothing much past one computer ( the browser ) asking another computer ( the server ) a question and receiving an answer.

In the first round at creating a blog, we created a static server that served files out of a public folder and we stored our blog inside of public/blog.

This worked for the first post but immediately became cumbersome for the next one. Copy, pasting, editing, removing, etc. What I really wanted and alluded to building was a way to write my blog posts in markdown, like this one is written in, and have my worker figure out how to make it into html.

We could have switch to WordPress or Gatsby or any other excellent blogging provider that offers plugins for such a thing but where’s the learning with that approach? If this was for a client? Sure, use the right tool for the job. But it’s just for me, so why not splurge some time on it?

baby steps

Let’s first write out in English what our worker will have to do in order to fulfill our idea

  1. Read a directory of files
  2. Transform each Markdown file into an HTMl file
  3. Save the HTML file into a specific directory

It sounds like I need to learn how to read a file and write a file in order to do anything that I’ll need to, so let’s first write that.

Suppose that we have a file text.txt

abc
def

We can read that file by creating a JS file index.js

// fs is the built-in Node filesystem module
const fs = require("fs");
// path is the built-in Node path module
const path = require("path");

fs.readFile(path.resolve(__dirname, "text.txt"), (err, data) => {
  if (err) {
    console.log(err);
    process.exit(1);
  } else {
    console.log(data);
  }
});

The above should print the following to the console

<Buffer <some weird bytes> ... <some number> more bytes>

that is because we did not give readFile an encoding for the file, causing it to return to us a buffer. We instead want our worker to return to us a string. We can do this by calling toString() on the Buffer or having our worker handle all that by passing in an encoding option to readFile

const fs = require("fs");
const path = require("path");

fs.readFile(path.resolve(__dirname, "text.txt"), "utf8", (err, data) => {
  if (err) {
    console.log(err);
    process.exit(1);
  } else {
    console.log(data);
  }
});

This should cause the worker to print the contents of text.txt to the console

abc
def

Now that we can read a file, let’s learn how to write a file. We can use fs for this as well!

const fs = require("fs");
const path = require("path");

fs.writeFile(path.resolve(__dirname, "other.txt"), "hello\nworld!", err => {
  if (err) {
    console.log(err);
    process.exit(1);
  } else {
    console.log("done!");
  }
});

which should have caused a new file, other.txt, to be created and have the contents of the string we passed in

hello
world!

Just like how our worker reads strings, when creating a string \n will be treated as a new line!

Real World

Now that we have the first Lego pieces built, we need to learn how to do the second part: transforming from Markdown to HTML. To do that, we are going to lean on others before us and install some NPM packages!

yarn init -y && yarn add markdown-it

markdown-it is our cheat code for transforming markdown to html. With it, we can simply point the markdown module to our readFile and get its output to writeFile

First, create some post.md

# Hello, world!

> This is my post!

and it's cool and stuff

- and things
  - and such

next, point our first Lego piece of reading a file, to that newly created post

const fs = require("fs");
const path = require("path");
const md = require("markdown-it")({
  html: true,
  linkify: true,
  typographer: true
});

fs.readFile(path.resolve(__dirname, "post.md"), "utf8", (err, data) => {
  if (err) {
    console.log(ee);
    process.exit(1);
  } else {
    // transform the data from markdown
    // into html
    const parsedFile = md.render(data);

    fs.writeFile(path.resolve(__dirname, "post.html"), parsedFile, err => {
      if (err) {
        console.log(err);
        process.exit(1);
      } else {
        console.log("done!");
      }
    });
  }
});

Running that should give us the following post.html file

<h1>Hello, world!</h1>
<blockquote><p>This is my post!</p></blockquote>
<p>and it’s cool and stuff</p>
<ul>
  <li>
    and things
    <ul>
      <li>and such</li>
    </ul>
  </li>
</ul>

But! That’s just the post. How can we add that HTML markup to some template that all posts will carry? Once again, an NPM package to the rescue!

yarn add cheerio

cheerio is a jQuery-like module that offers a way to manipulate an HTML string via a $-eqsue API. What we want to use it for is to take the HTML of some template.html file and add the HTML of our post.html to it. We can do that just by adding one more readFile and a new template.html file

<html>
  <head>
    <!-- common stuff -->
  </head>
  <body>
    <!-- some id to target in code -->
    <div id="post"></div>
  </body>
</html>
const fs = require("fs");
const path = require("path");
const md = require("markdown-it")({
  html: true,
  linkify: true,
  typographer: true
});
const cheerio = require("cheerio");

fs.readFile(
  path.resolve(__dirname, "template.html"),
  "utf8",
  (err, templateString) => {
    // Create a jQuery-like instance of the
    // template itself
    const $ = cheerio.load(templateString);

    // Then we read the post file
    fs.readFile(
      path.resolve(__dirname, "post.md"),
      "utf8",
      (err, postString) => {
        if (err) {
          console.log(err);
          process.exit(1);
        } else {
          // transform the data from markdown
          // into html
          const parsedFile = md.render(postString);

          // append the newly created parsedFile html
          // to the template via $
          $("#post").html(parsedFile);
          // and finally we get the full HTML
          // of the blog page
          const blogHTML = $.html();
          // and we write THAT to the disk as post.html
          fs.writeFile(path.resolve(__dirname, "post.html"), blogHTML, err => {
            if (err) {
              console.log(err);
              process.exit(1);
            } else {
              console.log("done!");
            }
          });
        }
      }
    );
  }
);

which will create a post.html file that is the template.html file with the post div containing the transformed markdown!

Final Form

So now we can transform markdown into a template of html, we have one last puzzle piece to solve: doing this for n files. We can solve this by reading the contents of the directory of blog posts and iterating over them!

const fs = require("fs");
const path = require("path");
const md = require("markdown-it")({
  html: true,
  linkify: true,
  typographer: true
});
const cheerio = require("cheerio");

fs.readFile(
  path.resolve(__dirname, "template.html"),
  "utf8",
  (err, templateString) => {
    // Create a jQuery-like instance of the
    // template itself
    const $ = cheerio.load(templateString);

    // Read all files inside of this directory
    fs.readdir(path.resolve(__dirname), (err, files) => {
      // Only get the files that end in markdown
      const posts = files.filter(f => f.indexOf(".md") > -1);

      // for each of those files
      posts.forEach(filename => {
        // read the post file
        fs.readFile(
          path.resolve(__dirname, filename),
          "utf8",
          (err, postString) => {
            if (err) {
              console.log(ee);
              process.exit(1);
            } else {
              // transform the data from markdown
              // into html
              const parsedFile = md.render(postString);

              // append the newly created parsedFile html
              // to the template via $
              $("#post").html(parsedFile);
              // and finally we get the full HTML
              // of the blog page
              const blogHTML = $.html();
              // and we write THAT to the disk as <filename>.html
              fs.writeFile(
                path.resolve(__dirname, `${filename}.html`),
                blogHTML,
                err => {
                  if (err) {
                    console.log(err);
                    process.exit(1);
                  } else {
                    console.log("done!");
                  }
                }
              );
            }
          }
        );
      });
    });
  }
);

but, but, CALLBACK HELL!

I know, it’s just so hard to read that. We need to use promises and async/await. Let’s make the above fancy, which is the way that I have set this project up to create its posts!

const fs = require("fs");
const path = require("path");
const { promisify } = require("util");

const md = require("markdown-it")({
  html: true,
  linkify: true,
  typographer: true
});

const cheerio = require("cheerio");

const readFile = promisify(fs.readFile);
const writeFile = promisify(fs.writeFile);
const readDir = promisify(fs.readdir);

const rootDir = path.resolve(__dirname);

const templatePath = path.resolve(rootDir, "blog", "template.html");

const directoryPath = path.resolve(rootDir, "blog");
const blogPath = path.resolve(rootDir, "public", "blog");

const createPosts = async ({ template, inputDir, outputDir }) => {
  const templateString = await readFile(template, "utf8");
  const contents = await readDir(inputDir);
  const posts = contents.filter(file => file.indexOf(".md") > -1);

  posts.forEach(async filename => {
    const postString = await readFile(path.resolve(inputDir, filename), "utf8");

    const $ = cheerio.load(templateString);
    const postHTML = md.render(postString);

    $("#markdown-content").html(postHTML);

    await writeFile(
      path.resolve(outputDir, filename.replace(".md", ".html")),
      $.html()
    );
  });
};

(async () => {
  try {
    await createPosts({
      template: templatePath,
      inputDir: directoryPath,
      outputDir: blogPath
    });
  } catch (e) {
    console.log(e);
  }
})();