A single-pass, zero-AST MDX-to-JSX compiler written in Odin.
Uses direct byte-scanning — no AST, no nodes, no tokens.
~400× faster than @mdx-js/mdx at P50 latency (17 µs vs 7169 µs for 1 KB files).
| Area | Status |
|---|---|
| Headings (ATX, setext) | ✅ |
| Paragraphs | ✅ |
| Inline formatting (bold, italic, strikethrough, code) | ✅ |
| Links (inline, reference, implicit) | ✅ |
| Images | ✅ |
| Lists (ordered, unordered, nested, tight, loose, task) | ✅ |
| Continuation lines in list items | ✅ |
Pipe escaping in tables (|) |
✅ |
| Tables (GFM, alignment) | ✅ |
| Blockquotes (nested) | ✅ |
| Code fences (with syntax highlighting) | ✅ |
| Thematic breaks | ✅ |
| Escaped characters | ✅ |
| HTML entities | ✅ |
| HTML block passthrough (incl. void elements) | ✅ |
| Line breaks | ✅ |
| JSX/JS passthrough (balanced delimiter scan) | ✅ |
| Frontmatter | ✅ |
| Footnotes (GFM) | ✅ |
Math (LaTeX $...$, $$...$$) |
✅ |
Emoji shortcodes (:word:) |
✅ |
Definition lists (term : def) |
✅ |
Autolink literals (GFM <url>, <email>) |
✅ |
Comments (JSX {/* */}, HTML <!-- -->) |
✅ |
| Reference links (CommonMark) | ✅ |
| Compile-time feature flags (10 features, zero cost when off) | ✅ |
| Plugin system (BeforeBlock/AfterBlock hooks) | ✅ |
| C/Rust FFI | ✅ |
Heading anchors (id attributes) |
🔜 Planned |
Alert/admonition blocks (> [!NOTE]) |
🔜 Planned |
Bare URL autolinking (without < >) |
🔜 Planned |
Unicode emoji conversion (:smile: → 😄) |
🔜 Planned |
| Mermaid diagram passthrough | 🔜 Planned |
Most compilers build an AST, transform it, then emit code. That's three passes, each requiring allocations, intermediate representations, and memory management. mdz does everything in one pass: scan the input byte-by-byte and emit JSX directly.
These are the architectural decisions that make mdz ~900× faster than @mdx-js/mdx from the start — before any micro-optimizations:
@mdx-js/mdx uses remark-parse to build a full mdast (Markdown AST), then walks it to generate JSX. This means:
- Allocating AST nodes for every element (headings, paragraphs, inlines)
- Storing parent-child relationships
- Walking the tree to emit output
- Freeing all allocations
mdz skips this entirely. The input is scanned once, and JSX is emitted directly. There are no nodes to allocate, link, or free.
The hardest part of MDX compilation is parsing JavaScript/JSX — balanced braces, arrow functions, template literals, JSX expressions, etc. @mdx-js/mdx uses micromark for markdown and then hands JSX/JS regions to a full JS parser.
mdz takes a different approach: it detects JSX elements and JS expressions by scanning for balanced delimiters (< >, { }, strings, etc.) and copies them through verbatim. No JS parsing. This is the single biggest speed advantage — mdz doesn't try to understand JavaScript, it just finds where it ends.
Trade-off: mdz won't catch JS syntax errors inside JSX blocks. If you need that, use a linter separately.
@mdx-js/mdx composes output through multiple layers (remark plugins → rehype plugins → string concatenation). Each layer may allocate and free intermediate strings.
mdz writes directly into one pre-sized [dynamic]byte buffer. The buffer starts at input × 2 capacity and grows at most a few times. There is exactly one output buffer allocation per compilation.
Instead of parsing line-by-line and maintaining parser state, mdz classifies each block by inspecting its first byte:
#→ heading (ATX)>→ blockquote-or1-9→ list`→ fence<→ JSX element or HTML{→ JS expressioniore→ import/export---→ thematic break (or setext H2 underline — resolved by pre-processor)
This is O(1) classification — no regex, no line-by-line state machine.
@mdx-js/mdx is built around a plugin system (remark + rehype). Every plugin wraps the AST, transforms it, and passes it on. Even with zero plugins, the infrastructure still runs.
mdz offers a plugin system that hooks directly into the single-pass scan — no AST to wrap, no tree to transform. Plugins register for specific block types at two hook points:
- BeforeBlock — intercept before processing; remove, replace, or let it pass
- AfterBlock — transform or annotate what was written
This gives you the extensibility of a plugin architecture at zero cost when no plugins are registered.
mdz only recognizes * for emphasis, not _. This eliminates ambiguity when scanning — the scanner doesn't need to check if _ starts emphasis or is part of an identifier. A small sacrifice in convenience, a real saving in scan complexity.
| Feature | Example | Output | Gated |
|---|---|---|---|
| Heading (ATX) | ## Title |
<h2>Title</h2> |
always |
| Heading (setext) | Title\n=== |
<h1>Title</h1> (also --- for H2) |
always |
| Paragraph | text |
<p>text</p> |
always |
| Bold | **bold** |
<strong>bold</strong> |
always |
| Italic | *italic* |
<em>italic</em> |
always |
| Bold+Italic | ***both*** |
<em><strong>both</strong></em> |
always |
| Strikethrough | ~~text~~ |
<del>text</del> |
always |
| Inline code | `code` |
<code>code</code> |
always |
| Autolink | <https://x.com> |
<a href="https://x.com">https://x.com</a> |
Autolink |
| Autolink email | <user@host.com> |
<a href="mailto:user@host.com">user@host.com</a> |
Autolink |
| Link | [text](url) |
<a href="url">text</a> |
always |
| Image |  |
<img src="url" alt="alt"/> |
always |
| Unordered list | - item |
<ul><li>item</li></ul> |
always |
| Task list (unchecked) | - [ ] task |
<li class="task-item"><input type="checkbox" disabled/> task</li> |
TaskList |
| Task list (checked) | - [x] task |
<li class="task-item"><input type="checkbox" disabled checked/> task</li> |
TaskList |
| Ordered list | 1. item |
<ol><li>item</li></ol> |
always |
| Blockquote | > text |
<blockquote>...</blockquote> |
always |
| Nested blockquote | > > deeper |
<blockquote><blockquote>...</blockquote></blockquote> |
always |
| Thematic break | --- |
<hr/> |
always |
| Footnote reference | [^1] |
<sup><a href="#fn:1" id="fnref:1">1</a></sup> |
Footnote |
| Footnote definition | [^1]: content |
Collected and rendered in <section class="footnotes"> at end |
Footnote |
| Reference link | [text][ref] with [ref]: url |
<a href="url">text</a> |
ReferenceLink |
| Reference link (implicit) | [text][] with [text]: url |
<a href="url">text</a> |
ReferenceLink |
| Pipe escaping in tables | | a | b | |
| a | b | → literal ` |
a |
| Escaped chars | \* |
literal * |
always |
| HTML entities | & |
& (passthrough); bare & → & |
always |
| HTML block passthrough | <div>text</div> |
<div>text</div> (verbatim, not re-parsed) |
always |
| HTML void elements | <br>, <hr>, <img> |
<br>, <hr>, <img> (pass through at block/inline) |
always |
| Line breaks | line1\nline2 |
line1<br/>line2 |
always |
In addition to ATX headings (# Title), mdz supports setext headings with underlined syntax:
Heading 1
=========
Heading 2
---------<h1>Heading 1</h1>
<h2>Heading 2</h2>Setext headings are converted to ATX syntax during a pre-processing phase. The underline must be 3+ repeated characters (= or -) on a line by itself (whitespace-only allowed). A 2-dash line (--) is not a valid underline. Frontmatter --- delimiters are correctly preserved and not treated as heading underlines.
| Left | Center | Right |
| :--- | :----: | ----: |
| a | b | c |<table>
<thead><tr><th style="text-align: left">Left</th><th style="text-align: center">Center</th><th style="text-align: right">Right</th></tr></thead>
<tbody><tr><td>a</td><td>b</td><td>c</td></tr></tbody>
</table>Alignment detection via :---, :---:, ---:. Gated by Tables feature flag (compile-time, default on).
Escaped pipes (\|) render as literal | inside cells instead of acting as cell separators:
| a \| b | c |
|---|---|
| d | e |<table>
<thead><tr><th>a | b</th><th>c</th></tr></thead>
<tbody><tr><td>d</td><td>e</td></tr></tbody>
</table>The escape \| is passed through to the inline processor which converts it to a literal pipe character. This follows the same escape semantics as the rest of the compiler.
Input:
```python
def fib(n):
return n if n < 2 else fib(n-1) + fib(n-2)
```Output:
<pre><code class="language-python"><span class="hl-kw">def</span> fib(n):
<span class="hl-kw">return</span> n <span class="hl-kw">if</span> n < <span class="hl-num">2</span> <span class="hl-kw">else</span> fib(n-<span class="hl-num">1</span>) + fib(n-<span class="hl-num">2</span>)
</code></pre>Supported languages: JS/TS/JSX/TSX, CSS/SCSS/LESS, HTML/XML/SVG, Python, Rust, Go, Bash/Zsh, JSON, YAML.
Token types highlighted: keywords (hl-kw), strings (hl-str), comments (hl-cm), numbers (hl-num).
Gated by Highlight feature flag (compile-time, default on). Unknown languages fall back to plain escaped output.
Inline: $E = mc^2$ → <code class="language-math">E = mc^2</code>
Display:
$$
\int_a^b f(x) \, dx
$$→ <pre><code class="language-math">\int_a^b f(x) \, dx</code></pre>
Multi-line display math survives the single-pass pipeline via placeholder substitution — $$...$$ regions are extracted during a pre-processing phase and restored after output assembly. $$ inside fenced code blocks is not treated as math.
Gated by Math feature flag (compile-time, default on).
:word: → <span class="emoji">word</span>
Emoji shortcodes are case-sensitive word characters (a-z, 0-9, _, -) between colons. Non-matching : renders as a literal character. Time patterns like 12:30 are not affected since the character after : must be a word character.
Gated by Emoji feature flag (compile-time, default on).
Strips both JSX-style and HTML-style comments at the block level:
| Syntax | Example |
|---|---|
| JSX comment | {/* this is hidden */} |
| HTML comment | <!-- this is hidden --> |
Comments are removed entirely from output — they do not produce any emitted JSX.
Gated by Comment feature flag (compile-time, default on).
Term
: Definition one
: Definition two<dl>
<dt>Term</dt>
<dd>Definition one</dd>
<dd>Definition two</dd>
</dl>The term and its definitions must be adjacent — a blank line between them separates them into independent blocks (per Markdown spec). Standalone : lines with no preceding term are silently skipped.
Gated by DefList feature flag (compile-time, default on).
URI autolinks: <https://example.com> → <a href="https://example.com">https://example.com</a>
Email autolinks: <user@example.com> → <a href="mailto:user@example.com">user@example.com</a>
Any letter-based URI scheme is detected (http://, https://, ftp://, irc://, mailto:, etc.). Autolinks take precedence over JSX element detection at both the block and inline levels — <https://...> is treated as a paragraph with an autolink, not a JSX element.
Gated by Autolink feature flag (compile-time, default on).
Both ordered (1.) and unordered (-, *) lists are supported, including arbitrary nesting via indentation:
- Outer item
- Nested item
- Another nested
- Back to outer<ul>
<li>Outer item
<ul>
<li>Nested item</li>
<li>Another nested</li>
</ul>
</li>
<li>Back to outer</li>
</ul>List items absorb continuation lines indented to at least the content column. Lines are joined with spaces for inline processing:
- This item has a
continuation line that
spans multiple lines<ul>
<li>This item has a continuation line that spans multiple lines</li>
</ul>Per CommonMark behavior: blank lines between list items make the entire list "loose," wrapping each item's content in <p> tags. Without blank lines, the list is "tight" and items render without paragraph wrappers:
- Tight item
- Another tight<ul>
<li>Tight item</li>
<li>Another tight</li>
</ul>- Loose item
- Another loose<ul>
<li>
<p>Loose item</p>
</li>
<li>
<p>Another loose</p>
</li>
</ul>- [ ] Unchecked task
- [x] Checked task
- [X] Checked (uppercase)<ul>
<li class="task-item"><input type="checkbox" disabled/> Unchecked task</li>
<li class="task-item"><input type="checkbox" disabled checked/> Checked task</li>
<li class="task-item"><input type="checkbox" disabled checked/> Checked (uppercase)</li>
</ul>Only - (hyphen-space) list markers are checked for task list syntax; * and ordered markers are not. Mixed regular and task items in the same list work correctly.
Gated by TaskList feature flag (compile-time, default on).
Here is a footnote reference[^1] and another[^longnote].
[^1]: This is the first footnote.
[^longnote]: This has more content.<p>Here is a footnote reference<sup><a href="#fn:1" id="fnref:1">1</a></sup> and another<sup><a href="#fn:longnote" id="fnref:longnote">longnote</a></sup></p>
<section class="footnotes">
<ol>
<li id="fn:1">This is the first footnote. <a href="#fnref:1" class="footnote-backref">↩</a></li>
<li id="fn:longnote">This has more content. <a href="#fnref:longnote" class="footnote-backref">↩</a></li>
</ol>
</section>Footnote definitions ([^id]: content) are stripped from the input stream during a pre-processing phase and collected into a list. After all body content is compiled, the footnotes section is appended at the end. Inline content within footnotes (bold, links, code, etc.) is processed normally.
Gated by Footnote feature flag (compile-time, default on).
Reference-style links allow you to define URLs separately and refer to them by label:
Here is [a reference][ref-label] and an [implicit][] one.
[ref-label]: https://example.com "Optional Title"
[implicit]: https://example.org<p>Here is <a href="https://example.com" title="Optional Title">a reference</a> and an <a href="https://example.org">implicit</a> one.</p>The [text][] form uses text as the implicit label. Labels are case-insensitive and collapse internal whitespace. Unresolved references (no matching definition) render as literal brackets [text][ref].
Reference definitions ([label]: url "title") are stripped from the output during a pre-processing phase. Definitions support bare URLs, angle-bracketed URLs (<url>), and titles in double quotes, single quotes, or parentheses.
Fenced code blocks are correctly skipped — reference patterns inside code fences are not replaced.
Gated by ReferenceLink feature flag (compile-time, default on).
All JSX elements and JS expressions are detected by balanced delimiter scanning and passed through verbatim — no JS parsing. This is the core MDX compatibility that makes mdz a drop-in replacement.
| Construct | Example |
|---|---|
| JSX element | <Component prop={value} /> |
| Multi-line JSX | <Component\n data={[1,2,3]}\n/> |
| Nested JSX | <Outer><Inner /></Outer> |
| JS expression | {items.map(i => <div key={i}>{i}</div>)} |
| Import | import {X} from './module' |
| Export | export const meta = { ... } |
| Module declarations | const x = 1, function foo() {} |
| Template literals | const x = `${name}` |
| Frontmatter | ---\ntitle: "Hello"\n--- |
The scanner handles string literals (', ", `), nested braces, JSX tag depth, multi-line self-closing attributes, and template literal interpolation (${...}). A safety bailout exits after 100 consecutive newlines inside an unclosed expression.
YAML frontmatter between --- markers at the start of the file is stripped from output. The frontmatter detector runs only at file position 0, so --- thematic breaks elsewhere in the document are not affected. The closing --- must be on its own line with optional trailing whitespace.
Raw HTML blocks are passed through to the output verbatim without re-parsing their content as Markdown. This includes block-level elements (<div>, <span>, <p>, <section>, etc.) as well as void elements that don't require closing tags in HTML:
| Void element | Self-closing equivalent |
|---|---|
<br> |
<br/> |
<hr> |
<hr/> |
<img src="..."> |
<img src="..."/> |
<input disabled> |
<input disabled/> |
<meta charset="utf-8"> |
<meta charset="utf-8"/> |
All 14 HTML void elements are recognized: area, base, br, col, embed, hr, img, input, link, meta, param, source, track, wbr.
At the block level, HTML blocks are detected by the leading < character and processed through the JSX block scanner. The depth tracker correctly handles nested elements (<div><p>text</p></div>) and self-closing/void elements (<br>, <img/>). Content after a blank line following an HTML block is parsed as normal Markdown.
Inline HTML elements (e.g., <span>text</span> inside a paragraph) are also passed through via inline JSX element detection.
All optional features can be disabled at compile time via Odin's -define mechanism, making the binary smaller and eliminating the associated code paths entirely:
odin build . -define:mdz.Tables=false -define:mdz.Math=false -define:mdz.Highlight=false| Flag | Controls | Default |
|---|---|---|
Tables |
GFM table parsing and rendering | true |
Math |
Inline $...$ and display $$...$$ |
true |
Highlight |
Syntax highlighting for code fences | true |
Emoji |
:word: → emoji spans |
true |
Comment |
Strip {/* */} and <!-- --> |
true |
DefList |
Definition list (term : def) |
true |
Autolink |
GFM autolink literals <url> and <email> |
true |
TaskList |
GFM task lists - [ ] / - [x] |
true |
Footnote |
GFM footnotes [^id] refs and definitions |
true |
ReferenceLink |
Reference-style links [text][ref] with [ref]: url |
true |
Disabled features compile out of the classification switch and processing dispatch — zero runtime overhead.
import mdz "mdz"
// Compile an MDX string to a JSX module string.
// The caller owns the returned string and should delete it.
result, err := mdz.compile(mdx_input)
if err != nil {
fmt.eprintf("error: %s\n", err.(mdz.Compile_Error).message)
}
defer delete(result)
// Or read from a file:
result, err = mdz.compile_file("input.mdx")Plugins hook into the single-pass compile loop at two points:
| Phase | Timing | Use case |
|---|---|---|
BeforeBlock |
After classification, before processing | Remove or replace blocks |
AfterBlock |
After processing | Transform or annotate output |
pipeline: mdz.PluginPipeline
mdz.plugin_init(&pipeline)
defer mdz.plugin_destroy(&pipeline)
// Remove all fences
mdz.plugin_register(&pipeline, mdz.Plugin{
name = "strip-fences",
phase = .BeforeBlock,
filter = {mdz.BlockType.Fence},
hook = proc(ctx: ^mdz.PluginContext) -> mdz.PluginResult {
return .Remove
},
})
// Annotate paragraphs
mdz.plugin_register(&pipeline, mdz.Plugin{
name = "annotate-p",
phase = .AfterBlock,
filter = {mdz.BlockType.Paragraph},
hook = proc(ctx: ^mdz.PluginContext) -> mdz.PluginResult {
mdz.writer_write(ctx.output, "<!-- p -->\n")
return .Keep
},
})
result, err := mdz.compile_with_plugins(input, nil, &pipeline)| PluginResult | Meaning |
|---|---|
.Keep |
Continue with normal processing |
.Remove |
Skip this block entirely |
.Handled |
Plugin wrote to output; skip normal processing |
The filter field limits the hook to specific block types. An empty filter fires on every block.
For simpler cases, TransformConfig provides per-block filter procs that return .Keep or .Remove:
config := mdz.TransformConfig{
heading = proc(input: string, pos: int) -> mdz.NodeAction {
return .Remove
},
}
result, err := mdz.compile_with(input, &config)Plugins and TransformConfig compose — config filters run first, then plugins.
mdz [options] <file.mdx>
Options:
-o <file>Write output to file instead of stdout--benchRun benchmarks-h, --helpShow help
Examples:
mdz input.mdx # print to stdout
mdz -o output.jsx input.mdx # write to file
mdz --bench # run benchmarks
Prerequisites: Odin nightly + a C compiler (Visual Studio on Windows, GCC/Clang on Linux/macOS).
make build # debug build
make build-fast # optimized build
make test # run testsbuild_fast.bat./build.sh-o:aggressive -microarch:native -no-bounds-check -disable-assert -lto:thin
~36% P50 improvement over -o:speed on 50 KB files (~1265 → 814 µs).
Benchmark methodology (inspired by Ahmed Ayob's article on P99 latency):
Generate N random MDX files (deterministic seed 42), warmup 1 file, measure per-file wall-clock latency.
Report P50 (median) and P99 (tail — what users feel). Throughput in MB/s of input processed.
| Config | Library | P50 | P99 | MB/s |
|---|---|---|---|---|
| 1KB × 50 | mdz | 18 µs | 25 µs | 56.1 |
| 1KB × 50 | @mdx-js/mdx | 7169 µs | 15126 µs | 0.13 |
| 10KB × 50 | mdz | 191 µs | 533 µs | 38.9 |
| 10KB × 50 | @mdx-js/mdx | 50712 µs | 63414 µs | 0.19 |
| 50KB × 50 | mdz | 912 µs | 2651 µs | 38.8 |
| 50KB × 50 | @mdx-js/mdx | 277285 µs | 328185 µs | 0.18 |
mdz is ~300–400× faster in P50 latency and ~200–300× higher throughput than @mdx-js/mdx.
Full benchmark details: benchmark/benchmark.md
| Config | P50 | P99 | Throughput |
|---|---|---|---|
| 1KB × 50 | 27 µs | 62 µs | 36.0 MB/s |
| 10KB × 50 | 445 µs | 798 µs | 22.6 MB/s |
| 50KB × 50 | 1700 µs | 3207 µs | 26.8 MB/s |
main.odin CLI — reads file, calls compile_file, prints output
mdz/
buffer.odin Writer — pre-allocated [dynamic]byte wrapper
types.odin BlockType enum, Compile_Error, TransformConfig
config.odin Compile-time feature flags (Tables, Math, Highlight, etc.)
scanner.odin Byte helpers, block classification, JS boundary detection
inline.odin Inline markdown→JSX (bold, italic, code, links, images, expressions, autolinks, footnotes)
blocks.odin Block-level processors (headings, lists, fences, blockquotes, task lists, etc.)
highlight.odin Syntax highlighting tokenizer for fenced code blocks
plugin.odin Plugin pipeline — BeforeBlock/AfterBlock hooks
mdz.odin Public API — compile / compile_with / compile_with_plugins
Input MDX
│
├── Math pre-process ──► Extract $$...$$, $...$ into placeholders
├── Setext pre-process ──► Convert === / --- underlines to # / ##
├── Footnote pre-process ──► Collect [^id]: definitions, strip from stream
├── RefLink pre-process ──► Collect [label]: url defs → replace [text][ref] with [text](url)
│ └── Early exit (return input) when 0 defs found
│
▼
classify_block ──► First byte inspection → BlockType
│
├── TransformConfig check ──► .Remove? skip
│
├── Plugin BeforeBlock ──► .Remove / .Handled / .Keep
│
▼
process_* ──► Dispatched by BlockType, emits JSX to Writer
│ (or skips frontmatter)
│
├── Plugin AfterBlock ──► Transform / annotate written output
│
├── Footnote post-process ──► Append <section class="footnotes"> to body
│
▼
Output assembly ──► module_code + body wrapped in MDXContent()
│
└── Math restore ──► Replace placeholders with <code class="language-math">
mdz can be compiled as a shared library and used from C or Rust.
make ffi # all platformsOr manually:
odin build ffi -build:shared -o:aggressive -no-bounds-check -disable-assertProduces mdz.dll (Windows), libmdz.so (Linux), or libmdz.dylib (macOS) in the project root.
#include "c/mdz.h"
char* output = NULL;
char* error = NULL;
int ret = mdz_compile_file("input.mdx", &output, &error);
if (ret == 0) {
printf("%s", output);
mdz_free_string(output);
} else {
fprintf(stderr, "%s\n", error);
mdz_free_string(error);
}Build and run:
make c-example # or: cd c && make
./c/mdz_c input.mdxuse mdz_sys;
let output = mdz_sys::compile_file("input.mdx").unwrap();
println!("{}", output);Run the Rust example:
make rust-example # or: cd rust && cargo run --example compile -- ../input.mdxRun both mdz and @mdx-js/mdx benchmarks:
make bench # mdz only
make bench-mdx # @mdx-js/mdx onlyOr use the scripts:
benchmark/run_benchmarks.sh # Linux/macOS
benchmark/run_benchmarks.bat # WindowsResults are saved to benchmark/mdz_results.txt and benchmark/mdx_results.txt.
The design decisions above made mdz fast from day one (~27 MB/s with no features). Here's how we took it from 27 MB/s to where it is today — a fully-featured compiler that still outruns the JavaScript ecosystem by ~400×.
Problem: convert_inline wrote every non-special character one byte at a time through writer_write_byte. For a 50 KB file of mostly plain text, that's ~50,000 individual append calls, each checking capacity.
Fix: Scan for the next trigger character (*, `, [, !, {, <, ~, \, &), then copy the entire span in one writer_write call. This converts O(n) appends into O(triggers) appends.
Gain: ~15% (27→31 MB/s).
Problem: writer_write iterated byte-by-byte with #no_bounds_check for i in 0..<len(s) { append(&buf, s[i]) }. Each iteration called append which checked capacity, created a full call frame, and resolved the dynamic array pointer. The loop overhead was massive.
Fix: Replace with a single append(&w.buf, ..transmute([]byte)s). This is a single memcpy — the runtime copies all bytes in one shot with no per-byte overhead.
Problem: process_paragraph called split_lines(content) which heap-allocated a []string array, then iterated it.
Fix: Scan for \n inline and call convert_inline per line segment directly. Zero allocation.
Problem: Buffer starting size input + 25% caused growth reallocations.
Fix: Use input × 2 as starting capacity for the body writer. Fewer reallocations = fewer memcpys of accumulated data.
Gain: ~3× (31→100 MB/s). The single biggest win.
Problem: write_escaped and write_url_escaped still wrote byte-by-byte in their default case (non-special chars).
Fix: Same pattern as Round 1 — find the next character that needs escaping, batch-copy the span.
Problem: handle_link scanned backward through the entire URL looking for a space before an optional title. For links without titles (the common case), this was wasted scanning.
Fix: Check the last character first — if it's not " or ', there's no title, skip the backward scan entirely.
Gain: ~30% (70→115 MB/s for 50 KB files).
Problem: The assembly phase created a result Writer, copied module + body + boilerplate into it, then strings.clone'd the result to return it. This was two full copies of the output.
Fix: Use strings.concatenate([]string{...}) instead. It pre-computes the total length, allocates one buffer, and copies each piece in one pass. No clone needed.
Gain: <5% (saved one memcpy, but processing dominates).
A systematic pass through every function:
| File | Change | Why |
|---|---|---|
scanner.odin |
Frontmatter scan only runs when pos==0 (file start) |
Avoids O(n) scan for closing --- on every --- thematic break in mid-file |
scanner.odin |
skip_blank_lines switch → if-chain |
Switch compiled to jump table; if-chain is simpler for 4 cases |
scanner.odin |
Removed dead is_within_fence |
Unused code |
inline.odin |
Inlined trigger check in convert_inline hot path |
Avoids function call/ret overhead per character |
inline.odin |
find_closing single-char fast path |
Skips substring comparison for *, ~, ` |
blocks.odin |
process_heading tag emission simplified |
Removed intermediate [3]byte array |
Gain: Small improvements lost in Windows scheduler noise (±5-10%), but real reductions in code path length.
Problem: Odin's -o:speed is the standard release flag, but Odin also offers -o:aggressive (use with caution) which enables additional LLVM optimization passes at the cost of longer compile time.
Fix: Switch all build targets from -o:speed to -o:aggressive. One character change in Makefile, build_fast.bat, build.sh, and ffi/build.bat.
Gain: ~36% P50 improvement on 50 KB files (1265 → 814 µs). Throughput: 43.9 → 51.2 MB/s (+17%). No stability issues — all tests pass, no runtime crashes. Larger files benefit the most; small files (<10 KB) are within noise.
Problem: Adding syntax highlighting for code fences meant tokenizing every code block. A naive implementation would add significant per-file cost.
Fix: Wire a line-oriented tokenizer (mdz/highlight.odin) directly into the single-pass pipeline. The tokenizer emits <span class="hl-*"> wrappers for keywords, strings, comments, and numbers — one pass, no intermediate allocation. The SyntaxBundle (grammar rules) is compiled into the binary; no grammar files to load at runtime.
Trade-off discovery: two numbers moved in opposite directions. The native-only benchmark got 4× slower (12 → 47 ms for 1000 files) because code blocks were now being highlighted in Rust instead of in a sidecar. The realistic benchmark (full plugin chain) dropped 3× (2666 → 886 ms). The same work moved from a JavaScript sidecar into the native pipeline — net gain for the user, net loss for the isolated native benchmark.
The highlighting cost is gated at three levels:
- Compile time: the
Highlightfeature flag can disable it entirely - Config time: if the language group is unknown, it falls back to plain
write_escaped— no tokenizer overhead - Per-file: code blocks without fences skip the highlighter entirely
This is why you need to read both numbers — if we'd only watched the native column, we'd have rolled back the right change for the wrong reason.
Problem: Users wanted the ability to intercept or transform the compiler's output without forking the codebase. A traditional plugin architecture (trait objects, dynamic dispatch) would add per-block overhead even when no plugins are registered.
Fix: The plugin pipeline is a ^PluginPipeline parameter that defaults to nil. The main loop checks if pipeline != nil once per block — when nil, the plugin hooks are dead code that the optimizer eliminates. When present, run_before_plugins and run_after_plugins iterate the registered plugins, each with a per-plugin filter check against the current BlockType.
if pipeline != nil {
result := run_before_plugins(pipeline, input, &pos, &body_w, &module_w, bt)
if result == .Remove { skip_removed_block(...); continue }
if result == .Handled { continue }
}Cost: When no plugins are registered, exactly one nil check per block — no dispatch, no allocation, no vtable. When plugins are active, only plugins whose filter matches the current block type execute. An empty pipeline produces byte-identical output to compile().
Problem: Ten optional features (Tables, Math, Highlight, Emoji, Comment, DefList, Autolink, TaskList, Footnote, ReferenceLink) each add classification branches, processing code, and output paths. Users who don't need them still pay the code size and branch overhead.
Fix: Use Odin's #config(name, default) mechanism in mdz/config.odin. Every feature defaults to true. At compile time, the user can disable any subset:
odin build . -define:mdz.Tables=false -define:mdz.Emoji=falseWhen a feature is disabled, three things happen:
classify_blockremoves that feature's type check from the first-byte switch- The
#partial switchin the compile loop omits that feature's case entirely - The associated
process_*function is never linked into the binary (~1-2 MB per feature)
Impact: A fully stripped binary (all features off) is ~4 MB vs ~12 MB with everything enabled. The runtime cost of a disabled feature is exactly zero — the code path doesn't exist in the binary.
-
Avoid per-byte operations. Every byte that goes through a function call, capacity check, or dynamic dispatch adds overhead. Batch everything you can.
-
Transmute
stringto[]bytefor bulk append.append(&buf, ..s)doesn't compile (string ≠ []byte), butappend(&buf, ..transmute([]byte)s)does one memcpy. This was the single biggest win. -
Pre-size buffers.
[dynamic]bytegrows exponentially (2×), but every growth copies existing data. Pre-size with a generous estimate. -
Avoid allocations in hot paths.
split_linesinprocess_paragraphwas trivially replaced with inline scanning, eliminating one heap allocation per paragraph. -
P99 matters more than P50. The median is the cost when nothing is wrong. P99 is the cost when the OS scheduler preempts you, a page fault occurs, or the CPU migrates cores. On Windows, P99 variance can be ±50 µs from scheduler noise alone.
-
Two numbers moving in opposite directions can both be correct. Adding syntax highlighting made the native benchmark 4× slower but the realistic plugin-chain benchmark 3× faster — the same work moved from JavaScript to native. If you only watch one number you'll roll back the right change for the wrong reason.
-
The first refactor often won't move the bench. The plugin system refactor (extracting
_compile, adding the pipeline) didn't improve throughput. But it unblocked the BeforeBlock/AfterBlock hook model that lets users intercept the compiler without forking it. Sometimes the prerequisite work is invisible in timing — trust the model. -
Compiler flags are free wins. Swapping
-o:speedfor-o:aggressiveis a one-character change that gave 36% P50 improvement on large files. Read your compiler's optimization flag documentation — there may be a level above what you're using. -
Compile-time gating beats runtime branching. Feature flags via
#configlet the optimizer eliminate entire code paths at link time. A disabled feature costs zero — no branch, no dead code, no struct field. This pattern works in any language with conditional compilation (Rustcfg, C#if, Odinwhen).
Problem: The reference link two-pass preprocessor scanned every byte in Phase 2 even when no reference definitions were found — the common case for most Markdown files. Each non-matching byte went through strings.write_byte, adding O(n) append overhead to every compilation.
Fix: Two changes to preprocess_reference_links in mdz/mdz.odin:
- Early exit — return the original
inputimmediately whenlen(defs) == 0. No Phase 2 allocation, no scan, no per-byte writes. - Bulk-range writes — for the remaining case (defs present), track a
range_startcursor and flush unchanged regions with a singlestrings.write_stringinstead of per-bytewrite_byte.
Gain: Zero measurable overhead for files without reference definitions (the 99% case). The feature is now cost-proportional-to-value — files without reference links pay nothing.
Problem: List items only captured one line (no continuation line support), blank lines between items were silently ignored, and nested lists had a closing-tag bug where the parent <li> was never closed after a nested </ul>.
Fix: Three structural changes to process_list in mdz/blocks.odin:
- Continuation lines — the first-pass scan now absorbs lines indented ≥ the content column, joining them into the item content.
- Blank line tracking — blank lines between items set an
is_looseflag; loose lists wrap item content in<p>tags during emit. - Nested list closing — added a
nested_closedflag so the parent<li>is properly closed when returning from an inner list level.
Gain: Correct CommonMark behavior for list items spanning multiple lines and loose/tight rendering. The \r character is treated as whitespace for Windows \r\n compatibility.
main.odin CLI entry point + benchmarks
mdz/ Core library
ffi/ Shared library (C ABI) wrapper
c/ C header + example
rust/ Rust crate (mdz-sys)
benchmark/ Benchmarks + comparison results
build.sh Unix build script
build_fast.bat Windows build script
Makefile Cross-platform build
.odin/ odin package index
test.odin Unit tests
MIT License — see LICENSE.