writr-rs: byte-exact Rust markdown engine (native + wasm + browser) with multi-core rendering - #487
Merged
Merged
Conversation
- Cargo workspace with writr-core (pure engine) and writr-conformance (reads test/harness goldens directly from Rust). - Byte-exact ports of: writr's frontmatter strip (body getter quirks included), mdast-util-to-hast@13.2.1 (all handlers incl. GFM/footnotes/ math shapes/custom MDX JSX), hast-util-to-html@9.0.5 defaults (stringify-entities hex references, property-information attribute mapping via code-generated tables), micromark normalize-uri and trim-lines. - Vendored markdown-rs 1.0.0 with a one-line to_mdast fix: leading virtual spaces from partially-consumed tabs were dropped (upstream to_html handles them; commonmark/0159 0168 0250 prove it). - Gate: commonmark profile 454/454 goldens byte-exact via cargo test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Ports remark-github-blockquote-alert@2.1.0 into the blockquote conversion: marker scanning with the plugin's isNext semantics, the multi-line strip vs single-line drop branches, injected title paragraph with exact octicon SVG path data, and string-valued className quirk. GFM tables/tasklists/strikethrough/autolinks/footnotes were already exact from the M1 handler ports. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
- remark-toc@9/mdast-util-toc@7.1.0 port: shared-slugger heading walk (nested headings advance dedupe counters), opening/closing section detection, tight nested-list generation with link unwrapping and footnote-reference dropping, root-children splice. - remark-emoji@5.0.2 port: node-emoji@2.2.0 table (codegen from emojilib@2.4.0), find-and-replace exec semantics including the false-match rescan-at-position+1 rule and splice-skip. - rehype-slug@6 port with github-slugger@2.0.0 strip-ranges codegen (evaluated per-codepoint against the real regex, 736 ranges). - Both no-highlight and no-math profiles (73-diagnostic set) green; default corpus at 782/1037 pre-highlight/math. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
- hast-util-raw@9.1.0 port: token replay through html5ever's tree builder with a fresh tokenizer per raw chunk (seeded with tracked state + last start tag), parse5's template fragment context, and the parse5 table-text quirk where synthesized whitespace batches are foster-parented out of tables (token-type vs content classification). - hast-util-from-parse5@8/hastscript ports: attribute→property parsing (space/comma lists, JS Number() coercion incl. hex/exponent, boolean normalization), prototype-pollution name guard, SVG case adjustment, template content. - mdast-util-gfm-autolink-literal@2 transforms port: the parse-time find-and-replace that links URLs/emails across character escapes (micromark unicode class tables code-generated per UTF-16 unit). - rawhtml profile: 482/487; the 5 remaining are rehype-highlight's hljs classes (M4). mdx profile green as a side effect of the M1 handler ports — writr's custom JSX handler + expression/ESM text fallbacks were already exact. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
…xact
writr-hljs crate: full port of the hljs tokenizer over fancy-regex —
mode compiler with every compiler extension (scopeClassName, match,
MultiClass with group remapping, beforeMatch, beginKeywords, illegal,
relevance), ResumableMultiRegex with resume-at-same-position semantics,
keyword classification with MAX_KEYWORD_HITS relevance, sub-languages
(string + highlightAuto over subsets with illegal-abort partial trees),
END_SAME_AS_BEGIN/shebang/JSX-heuristic callbacks, and lowlight's
HastEmitter with dot-scope class conversion.
Grammars: all 36 lowlight-common languages serialized from the real
highlight.js as identity-preserving object arenas (cycles, shared
modes); callbacks canonicalized by function source (5 distinct).
JS-regex translation: ASCII \b/\w/\d via lookarounds/classes, JS dot
and \s sets, \uHHHH→\x{...}, \p{...} payloads, JS empty classes,
quantifier-vs-literal braces, capture-group counting and hljs's
backreference renumbering.
Fixture oracle: 2,111 corpus/diagnostic fences rendered through the
real lowlight@3.3.0 (tools/gen-hljs-fixtures.mjs) — all byte-exact.
Also fixes surfaced by the corpus: rehype-highlight glue (hljs class
before registration check, empty-language falsiness), serde_json
preserve_order (JS object-order keyword precedence), and three more
vendored markdown-rs parity patches: micromark's previousUnbalanced
guard for autolink literals, no mailto:/xmpp: email prefixes, trailing
blank lines keeping lists tight, and flow byte-shortcuts falling
through to the gfm table like micromark's null hooks (table rows
starting with *, _, backticks, #, etc).
Default profile: 1034/1037 — only the three KaTeX math diagnostics
remain (M5). All other profiles fully green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
writr-katex: the real katex.min.js (vendored from katex@0.16.45, MIT) on an embedded QuickJS (rquickjs), replicating rehype-katex@7.0.1's exact call sequence (throwOnError, then strict:'ignore' retry, then katex-error span from the first error). Per-thread contexts with a 4 MiB stack for KaTeX's recursive parser; process-wide (formula, displayMode) memoization. writr-core hast/katex.rs: rehype-katex port — language-math / math-display / math-inline detection, code-in-pre scope promotion, fragment-parsing rendered markup through the html5ever machinery (hast-util-from-html-isomorphic equivalent), splice + SKIP. Also fixes html5ever's empty (not absent) prefix on adjusted foreign attributes (xmlns) which serialized as ':xmlns'. Conformance: all 7 harness profiles — commonmark, gfm-only, no-highlight, no-math, rawhtml, mdx, default — are now 100% byte-exact across all 2,041 goldens with an empty allowlist. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
writr-node exposes render/renderAsync/validate/renderToMdast/engineVersion through napi-rs v3. The harness adapter registers as HARNESS_ENGINE=writr-rust; all 2,041 goldens pass byte-exact natively and under WRITR_RS_FORCE_WASM=1. The wasm build targets single-threaded wasm32-wasip1: build.rs mirrors napi-build's wasi link recipe minus the -threads assumptions (emnapi-basic instead of emnapi-basic-mt, no async-worker exports, module-owned memory). That also makes the browser endpoint (browser.js) run without SharedArrayBuffer or COOP/COEP headers — verified in headless Chromium via tools/browser-smoke.mjs, byte-comparing eleven feature-heavy documents against the native binding, KaTeX-in-QuickJS included. pnpm build:rs / build:rs:wasm produce the artifacts the index.js loader resolves (napi platform names, plain writr-node.node, cargo fallback). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
… engine in the Node ecosystem Coverage: workspace at 97.75% lines (gate at 97, writr-node excluded as host-bound; harness covers it end-to-end). ~130 new tests including 248 byte-exact oracle cases captured from the real JS engine; every remaining uncovered line is documented as unreachable-by-construction in COVERAGE.md. Edge-case divergences found by oracle testing on inputs outside the golden corpus are recorded in KNOWN-DIVERGENCES.md rather than silently shipped. Performance (all gated by the 2,041-golden + 2,111-fixture byte-exact suites, which stayed green through every change): - vendor/markdown: single-pass EditMap::consume, bulk data-run consumption clamped to feed chunks, construct-aware data markers, ASCII classify table, inlined move_one fast path, pre-sized event buffers — ~39% faster than upstream markdown-rs 1.0.0 on the benchmark corpus (VENDORED.md). - writr-hljs: dual regex backend (regex crate for non-fancy patterns, fancy-regex fallback), per-rule race with span memoization, non-fancy start-superset prefilters in front of the backtracking VM, span-only keyword scanning, FxHashMap — ~3.4x faster tokenization. - writr-core: KaTeX parse memo (hast fragments), direct-to-buffer HTML serialization, and renderBatch — rayon-parallel multi-document rendering exposed through the napi binding (renderBatch/renderBatchAsync). benchmark/benchmark-rust.ts (4 shared vCPUs): renderBatch sustains ~31K docs/s minimal-profile vs markdown-it ~24K and marked ~19.5K single-thread loops on the identical corpus — and ~9.9K docs/s with the full default pipeline (highlight+math+toc+slug+emoji+gfm) that those libraries cannot express. Single-doc sync renders are ~5-6x writr-JS on every profile. CI: .github/workflows/writr-rs.yml — fmt, clippy -D warnings, tests, coverage gate, codegen-freshness (regenerate-and-diff), harness parity (native + wasm + Chromium browser smoke + untouched JS suite), and a native build matrix. Criterion stage benches in writr-core. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub. |
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
The binding package's lockfile pinned @napi-rs/tar@1.1.1 (a @napi-rs/cli transitive) published inside the repo's 48h minimumReleaseAge window, failing harness-parity's install step. The CLI was only used once to scaffold the loader templates — the wasm build is plain cargo (tools/build-wasm.mjs) — so remove it and shrink the binding's dev tree to @napi-rs/wasm-runtime + emnapi + playwright-core. Also set persist-credentials: false on all checkout steps in the writr-rs workflow (no job pushes back to the repo), per review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces writr-rs, a high-performance Rust implementation of the writr markdown engine that compiles to a native Node.js addon and WebAssembly, reproducing the JS unified/remark/rehype pipeline byte-for-byte. The review feedback is highly constructive, identifying critical bugs and optimization opportunities. Specifically, it points out potential UTF-8 slicing panics in compile.rs when handling escaped multi-byte characters, and translation bugs in regex_js.rs regarding the conditional behavior of \p and \k escapes. Additionally, the feedback recommends using conditional compilation attributes (#[cfg(feature = ...)]) across several modules and tables (such as emoji and slug) to ensure that disabled features are completely compiled out to prevent binary bloat.
…e gates, CI browser-smoke deps - writr-hljs rewrite_backreferences: decode escaped characters instead of two-byte slices — an escaped multi-byte char (e.g. \é) in a future grammar would have panicked on a UTF-8 boundary. Regression tests added. - writr-hljs regex translation: \p/\P are property escapes only under the u flag and bare \k is an identity escape (JS annex B) — previously both passed through unconditionally, which could misparse or fail compilation for future grammars. All 2,111 hljs fixtures unchanged. - writr-core: the emoji/gfm/toc/slug modules, their generated tables, and the shared slugger are now compiled out when their features are off (slugger and slug_ranges gate on any(slug, toc)); pipeline call sites cfg-gated. check_features still fails loudly when a disabled feature's runtime flag is set. New CI step compile-checks the whole feature matrix. - browser smoke: esbuild is now a devDependency of crates/writr-node with its bin resolved locally — a fresh CI install exposed no transitive esbuild bin, which failed harness-parity's browser step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
pnpm 11 (CI) no longer reads pnpm.onlyBuiltDependencies from package.json and hard-fails on esbuild's ignored build script. Move the allow-list to a local pnpm-workspace.yaml (allowBuilds, same syntax the repo root uses), which also makes the directory its own settings boundary — so the workflow installs without --ignore-workspace. Verified with pnpm@11.6.0 locally: install, esbuild bin, emnapi archive, wasi loader, and browser smoke all green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b676fdeab3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…, batch exports, CI trigger paths - from_mdast: repeated JSX attributes now follow the JS handler's plain- object semantics — first position, last value — instead of emitting duplicate HTML attributes (oracle-verified against the JS engine; regression test added). All 2,041 goldens unchanged. - Browser entry renamed to browser.mjs / writr-node.wasi-browser.mjs so it parses as ESM despite the package's "type": "commonjs" (Chromium smoke re-verified), and both wasm loaders now export renderBatch / renderBatchAsync alongside the scalar APIs. - writr-rs workflow also triggers on package.json / pnpm-lock.yaml so dependency bumps re-run codegen-freshness and harness parity. The TOC finding was checked against the JS engine and is not a divergence: writr's remark-toc configuration produces no list for a top-level TOC heading followed only by deeper headings, byte-identical in both engines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Two serializer micro-optimizations, byte-identical output on all 2,041 goldens and 2,111 hljs fixtures: - serialize_attribute_into writes each attribute directly into the output String (name + escaped value), replacing the per-element Vec<String> + join and the per-attribute String allocation. Attribute values now pass through as Cow, so unescaped values are written without a copy. This is most visible on highlighted code where every hljs span carries a class attribute. - to_html_with_capacity lets the pipeline seed the output buffer at input.len() * 1.25 + 64, avoiding growth reallocations for typical documents. Profile delta (101-doc corpus, 50 iters): hast+serialize stage ~15.7µs → ~12.5-14.9µs per doc on the minimal profile. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Byte-identical on all 2,111 hljs fixtures and 2,041 goldens: - do_end_match copied code[event.index..] into a fresh String on every end match — O(remaining source) per event, quadratic on long blocks. end_of_mode only needs a borrowed slice. - The run loop and do_begin/do_end materialized the inter-event gap and every lexeme as owned Strings; all are now borrowed slices of the code or the match event. - process_keywords segments the mode buffer into byte spans instead of accumulating per-segment Strings: plain runs between keyword emissions are contiguous, so each piece of text is copied exactly once, straight into the emitter. Keyword-hit counting no longer clones the word on repeat hits, and the buffer allocation is handed back for reuse. Profile delta (101-doc corpus, 50 iters): minimal+highlight 369µs → ~312-327µs per doc; default profile 380µs → ~357-360µs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
renderBatchBuffer / renderBatchBufferAsync take one UTF-8 buffer plus
n+1 boundary offsets and return the rendered HTML in the same packed
shape ({ html: Buffer, offsets: Uint32Array }). One V8→native handoff
each way and no per-document JS string materialization on the main
thread — aimed at byte pipelines (docs read from disk, HTML written
back to disk) and at removing the serial marshalling fraction that
caps renderBatch scaling as core count grows.
render_batch in writr-core is generalized to AsRef<str> inputs so the
binding can pass borrowed slices of the packed buffer straight into the
parallel renderer. Invalid ranges and non-UTF-8 documents throw; batch
output over 4 GiB is rejected explicitly.
On this container's 4 shared vCPUs the packed path measures ~2.5%
faster than renderBatch on pre-packed input; the win is architectural
(serial fraction removal) rather than dramatic at this core count.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
build-wasm.mjs now post-optimizes the module with binaryen's wasm-opt when available (graceful fallback to a plain copy otherwise) and compiles with -C target-feature=+simd128 by default — SIMD128 is baseline wasm in every current engine (Node >= 16.4, Chrome 91, Firefox 89, Safari 16.4); WRITR_WASM_NO_SIMD=1 opts out for older embedders. CI installs binaryen so the harness tests the module we actually ship. Measured on the 101-doc corpus via WRITR_RS_FORCE_WASM=1: minimal profile 166.7 → 157.6µs/doc, default profile 543.8 → 492.5µs/doc (wasm-opt and simd contribute roughly 2:1), module size 4.78 → 4.12 MB. Harness (2,041 goldens) and browser smoke stay green on the optimized module. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
tools/build-pgo.mjs instruments writr-core's profile example, trains it on the benchmark corpus across every feature profile, merges the profile data with the toolchain's llvm-profdata, and rebuilds the addon with -Cprofile-use (own target dir, so the regular build cache is untouched). copy-node-artifact.mjs learns WRITR_RS_TARGET_DIR to install from that directory. Measured: ~2-5% on the feature-heavy profiles (highlight/default), noise-level on minimal — real but not worth doubling every local build, hence opt-in rather than folded into build:rs. The PGO addon stays byte-exact on all 2,041 goldens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #487 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 5 5
Lines 518 518
Branches 144 144
=========================================
Hits 518 518 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Vendor patch (documented in VENDORED.md): Point's line/column/index/vs and Link's previous/next are now u32, halving Event and with it the memory traffic of every event push, EditMap splice, and resolver walk. Documents are capped at 4 GiB of markdown, far beyond practical input; Point::offset() is the usize accessor for byte slicing, and the public unist/mdast API keeps usize everywhere, so writr-core is untouched. Byte-exact on all 2,041 goldens and 2,111 hljs fixtures. Profile delta (101-doc corpus, 50 iters): parse_to_mdast 89-92 -> ~84.5µs/doc, minimal render ~103-106 -> ~97-102µs/doc, default ~357-364 -> ~333-341µs. Also: the profile example takes PROFILE_STAGES (comma-separated label filter) so callgrind runs can scope to a single pipeline configuration. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
stringify_entities walked every char, linearly probed the subset per char, allocated a String per call, and a format! String per escaped character — ~4% of a minimal render per callgrind. The new stringify_entities_into writes straight into the output buffer: every escape subset is pure ASCII, so clean runs are located with a byte scan and appended in bulk, and the &#xNN; escape is emitted with a two-digit hex table instead of format!. The pub stringify_entities wrapper stays for tests/compat. Byte-exact on all 2,041 goldens and 2,111 hljs fixtures; minimal render ~97-103 -> ~96-97µs/doc on the 101-doc corpus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Vendor micro-patch: serialize is called once per data event during tree compilation; it allocated prefix/suffix Strings and ran three-argument format! even in the common no-virtual-spaces case. Now it pushes into one pre-sized String. Byte-exact on all 2,041 goldens and 2,111 hljs fixtures; measures within noise on the corpus (kept for the strictly smaller instruction count on a per-text-node path). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Whole-corpus throughput is now ~34K docs/s via renderBatchBuffer (~33K via renderBatch) vs markdown-it ~22K and marked ~15K on the same 4 shared vCPUs; single-doc default profile ~4K ops/s. The vendored parser measures ~1.6x upstream markdown-rs 1.0.0 interleaved on the same container (u32 events included), and the bare-CommonMark gap vs markdown-it is now ~2x (was ~3x), with the remaining architectural trade and the next step (mdast-stage fusion) documented. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please check if the PR fulfills these requirements
writr-rs/COVERAGE.md;writr-nodeis host-bound and covered end-to-end by the harness. The JS suite's 100% gate is untouched and green.)What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Feature —
writr-rs, a Rust implementation of writr's markdown engine that reproduces the JS unified/remark/rehype pipeline byte-for-byte, shipped as a native Node addon, a WebAssembly module, and a browser ESM entry point. The JS engine and public API are untouched (src/**has zero changes); the only JS-side changes are the harness adapter registration, build scripts, and a new benchmark.What's inside
writr-rs/Cargo workspace (~15K lines of Rust + generated tables):writr-core— frontmatter, markdown-rs parsing, mdast transforms (GFM alerts, toc, emoji, autolink-literal fixups), hast pipeline (raw HTML via html5ever replay, slugs, highlight, KaTeX), and a hast-util-to-html@9-exact serializer.writr-hljs— a port of highlight.js 11.11.1's engine (compiler, resumable multi-regex, keyword classifier, emitter) plus 36 grammars generated from the real hljs package.writr-katex— the realkatex.min.js0.16.45 embedded in QuickJS for byte-exact math.writr-conformance— renders the repo's golden harness from Rust and byte-diffs.writr-node— napi-rs v3 bindings:render,renderAsync,renderBatch(multi-core),renderBatchAsync,renderBatchBuffer/renderBatchBufferAsync(packed bytes-in/bytes-out batches),validate,renderToMdast,engineVersion, plus wasm + browser loaders.vendor/markdown— vendored markdown-rs 1.0.0 with parity + performance patches, all documented inVENDORED.md.Byte-exact parity (the migration safety net)
HARNESS_ENGINE=writr-rust)WRITR_RS_FORCE_WASM=1)browser.js, no COOP/COEP needed)pnpm test(the JS suite) is untouched and green. Every performance commit below was individually gated on these suites staying byte-exact.Performance
Benchmarked with
benchmark/benchmark-rust.ts(tinybench, same corpus as the existing benchmarks; 4 shared vCPUs — gaps widen with real cores):Whole-corpus throughput (101 docs/call):
renderBatchBuffer(bytes in/out, all cores)renderBatch(all cores)Single-document latency: writr-rs sync is ~5–6.5× writr-JS per call on every profile, uncached, with a synchronous API (~4,000 ops/s on the full default pipeline vs ~565 for writr-JS). With writr's cache on top, repeat renders stay in the millions of ops/s.
The dedicated optimization pass (last 8 commits) cut the default profile another ~13%, the highlight-heavy profiles ~15%, and the wasm build ~9% on top of the original engine, via: streaming serializer (attributes + entity escaping written straight into a capacity-seeded buffer), a copy-free hljs hot loop (an O(n²) remainder copy eliminated, span-based keyword segmentation), 40-byte
u32parser events (halving event memory traffic; vendored parser now ~1.6× upstream markdown-rs 1.0.0),wasm-opt -O3+ SIMD128 for the wasm artifact (−14% module size), the packedrenderBatchBufferAPI (removes the serial marshalling fraction as cores grow), and an opt-in PGO build (pnpm build:rs:pgo, +2–5%).Coverage & CI
cargo llvm-covline-coverage gate for the workspace (97.75% lines at introduction, CI gate at 97;writr-noderequires a Node host and is covered end-to-end by the harness runs). Every uncovered line is documented as unreachable-by-construction inwritr-rs/COVERAGE.md. The suites include 248 byte-exact oracle cases captured from the real JS engine plus ~130 new targeted unit/integration tests.writr-rs/KNOWN-DIVERGENCES.mdas follow-up candidates rather than silently shipped..github/workflows/writr-rs.yml: fmt + clippy-D warnings+ tests + coverage gate, codegen-freshness check (regenerates all tables from the pinned npm packages and diffs), harness parity (native, wasm, browser smoke, JS suite), and a native build matrix.cargo bench -p writr-core).How to try it
🤖 Generated with Claude Code
https://claude.ai/code/session_01U2S9pW3ucDuNmi2DNu1WiJ
Generated by Claude Code