Markdown Exit Breaks the Rules With Async Rendering

This article is the continuation of Erase CLS by Automatically Optimizing Images with Vite and focuses on a more concrete use case: automatically optimizing images in Markdown files.

It is highly technical, and unless you're dealing with Markdown files, this could be a little abstract. Nevertheless, I highly encourage you to read both the previous and the current article, as it could give you some nice ideas.

Why write a dedicated article about it? Because, until now, this wasn't possible. The main Markdown parser in the JavaScript world is markdown-it, and it is fully synchronous.

This might not be an issue, but it means that all rules must be synchronous as well. You can't have asynchronous rules. You can't perform a fetch within a rule, and you can't parallelize anything.

A little recap is needed to make sure we're all on the same page.

Writing a Markdown Parser ​

I won't go into details, but essentially, the parser will split the Markdown text into tokens and then apply rules, which are basically functions, to those tokens to generate the HTML.

For example, the following text **bold text** will be tokenized into: ['strong_open', 'text', 'strong_close'] and the corresponding rules will generate the HTML: <strong>bold text</strong>. The token strong_open will be processed by the strong_open rule, which returns <strong>, and so on.

The rule we've been talking about since the beginning transforms a token into its corresponding HTML representation. The interesting part is that you can attach your own rules to the parser to extend its functionality.

Imagine you want to automatically apply the loading="lazy" attribute to all images in your Markdown files. You could write a rule that targets the image rule.

md.use((md) => {
  const imageRule = md.renderer.rules.image!
  md.renderer.rules.image = (tokens, idx, options, env, self) => {
    const token = tokens[idx]

    token.attrSet('loading', 'lazy') 

    return imageRule(tokens, idx, options, env, self)
  }
})

Now, the following Markdown ![alt text](image.jpg) will be rendered as <img src="image.jpg" alt="alt text" loading="lazy" />. This is a dream: automatically apply changes to files without touching them.

Synchronous Limitations ​

This is a dream until you realize that all of this happens synchronously. Rules are applied one by one, without any way to parallelize the process. For the highlighting rule, this can significantly reduce performance because you can't render all the code blocks at once—it's one at a time.

Apart from parallelizing the process, synchronous rules mean that you can't perform any asynchronous operations within a rule. For example, you can't fetch data from an API. This can quickly become limiting.

But, with markdown-exit, a drop-in replacement and complete rewrite of markdown-it using TypeScript, this is now possible. Asynchronous rules are now a reality.

Optimizing Images on the Fly ​

In the previous article, we leveraged a Vite plugin to automatically apply optimizations to images in our HTML. This was a good and useful start.

For me, this wasn't enough because I mostly deal with Markdown files, and building a custom rule is a better choice. That's what we'll see now.

All the source code is available on GitHub: Barbapapazes/markdown-exit-unpic. To keep this article simple, I won't show unrelated code like the project setup.

Let's imagine we have a src/index.ts file. This file is our main script, and the goal is to render raw Markdown to HTML while optimizing images.

We want to run the following command:

node src/index.ts

with this Markdown content:

# Hello World

![This is a placeholder](https://images.unsplash.com/photo-1762707826575-d48c35b9b666)

and have the following output:

<h1>Hello World</h1>
<p><img src="https://images.unsplash.com/photo-1762707826575-d48c35b9b666?auto=format&amp;fit=crop&amp;q=80&amp;w=800" alt="This is a placeholder" loading="lazy" width="2567" height="2000" style="background-size: cover; background-image: url(data:image/bmp;base64,Qk32BAAAAAAAADYAAAAoAAAACAAAAAgAAAABABgAAAAAAMAAAAATCwAAEwsAAAAAAAAAAAAACQYICgcIDAoKEQ8PFBISFRISEQ4OCgYGCAYGDAoKExESGhgYHRsbHBoaFhMTDAgICAcGDgwMGBcXIB8fIyIiISAgGhgYDgoLCQcGDw0NGRgYIiEhJiUlJCIiHBoaDwsMCgcHDgwMFxYWHx4eIyIiISAgGhgYDgsLCQYGDAkJEhAQGBcXGxoaGxkZFRMTDAgICAUECQYGDAkJEA0OEhAQEhAQDwwMCQYFBwUEBwUECQYGCwcJDQoKDQsKCwgHCQUE);"></p>

Building a Custom Async Rule ​

The core logic of the rule won't differ from what we've built in the Vite plugin. Instead of reading the file from the local filesystem, we will fetch it from Unsplash to simulate a remote bucket. Then, we will retrieve the width and height, and generate the blurhash. At the end, we will also apply some custom query parameters to avoid the client loading the full image, like you can do with Cloudflare Images.

First, we need to create a new markdown-exit instance and render the Markdown. This is really straightforward.

import { createMarkdownExit } from 'markdown-exit'

const md = createMarkdownExit()

async function render() {
  const html = await md.renderAsync(`# Hello World

![This is a placeholder](https://images.unsplash.com/photo-1762707826575-d48c35b9b666)`)

  console.log(html)
}

render()

Nothing fancy in this script. However, the output is the following:

<h1>Hello World</h1>
<p><img src="https://images.unsplash.com/photo-1762707826575-d48c35b9b666" alt="This is a placeholder"></p>

There are some missing parts, but that's a good start.

To change the image rendering behavior, we will hook into the image rule, adjust the token to our needs, and use the default implementation to generate the output.

To create a new rule, we can use the use function. It's like creating a Vue plugin. Then, bind the function we want to the image rule.

import type { MarkdownExit } from 'markdown-exit'
import { createMarkdownExit } from 'markdown-exit'

const md = createMarkdownExit()

md.use((md: MarkdownExit) => {
  const imageRule = md.renderer.rules.image!
  md.renderer.rules.image = async (tokens, idx, options, env, self) => {
    return imageRule(tokens, idx, options, env, self)
  }
})

async function render() {
  // ...
}

render()

Our function does nothing apart from calling the original image rule, which is already a good step because we don't want to reimplement it ourselves.

Then, we can grab the current token and try to add an attribute to it.

The tokens variable contains all available tokens. It's an array of tokens. To access the current one, we need to use the idx variable, which is the current index of the token.

const token = tokens[idx]

Now, we can modify the token. For example, we can add a loading attribute with the value lazy.

md.renderer.rules.image = async (tokens, idx, options, env, self) => {
  const token = tokens[idx]

  token.attrSet('loading', 'lazy')

  return imageRule(tokens, idx, options, env, self)
}

The output of our script is now the following:

<h1>Hello World</h1>
<p><img src="https://images.unsplash.com/photo-1762707826575-d48c35b9b666" alt="This is a placeholder" loading="lazy"></p>

Easy, right?

Based on our expected behavior, we need to add the width, height, blurhash, and some query parameters to the image. To do so, we will use the same logic as in Erase CLS by Automatically Optimizing Images with Vite, except that we will fetch the image from Unsplash, as you could do with any remote bucket.

const img = await fetch(src).then(res => res.bytes())
const data = await getPixels(img)

const blurhash = encode(Uint8ClampedArray.from(data.data), data.width, data.height, 4, 4)

Now, we can set the width, height, and style attributes on the token.

import { blurhashToDataUri } from '@unpic/placeholder'

token.attrSet('width', data.width.toString())
token.attrSet('height', data.height.toString())
token.attrSet('style', `background-size: cover; background-image: url(${blurhashToDataUri(blurhash)});`)

Finally, we can adjust the src attribute to add some query parameters.

token.attrSet('src', `${src}?auto=format&fit=crop&q=80&w=800`)

These queries are specific to Unsplash, but you could do the same with Cloudflare Images or any image optimization service.

A quick note on this approach: it works fine. Parallelism slightly improves performance because most of the computation happens in the encode function that transforms the image into a blurhash synchronously. On my computer, fetching the image takes about 100ms, while encoding it takes about 3 seconds.

Going Further ​

The current implementation is simple, and that's intentional. Now that you know how to optimize your images and create custom rules, you can easily adapt this approach to your needs, and even create new ones that fit your specific use cases.

To deal with the performance bottleneck, you have many possibilities. Three of them are:

1. Local pre-processing and local storage: If you store images within your code source, you can create a script that will go through all images and generate a JSON file containing the width, height, and the blurhash. Then, in the rule, you'll just have to read this file, and no more computation is needed. This will reduce the build time as well.
2. Local pre-processing and remote storage: If your images are stored remotely, you can create a similar script that will fetch the images, process them, and generate the JSON file. This file can be pushed to a bucket for later use. In the rule, instead of reading a file, you can fetch it from the bucket. Without asynchronous rules, this wasn't possible!
3. Remote pre-processing and remote storage: If your images are stored remotely, you can create a workflow that will be triggered each time a new file is added. The workflow generates the JSON file and stores it in the same bucket. In the rule, you can fetch the JSON file from the bucket and use it directly. The maximal optimization. With R2 notifications and Queues, this is simple to implement.

You could also create a worker pool using Node threads, but it is overkill for this use case.

I'm sure there are a lot of other ways to automate image optimization. Now, it's up to you to find the best approach that fits your workflow.

Hope you enjoyed the article! If you implement something similar, keep me posted!