bench: per-fixture tables with generator, aligned event surfaces, large-document benchmark - #27
Merged
Merged
Conversation
Each section opens with an overview ranking libraries by geometric mean across its fixtures (with the missing-fixture dominance rule and a vs-fastest column), followed by one table per fixture sorted by op/s with each library's speedup relative to the fixture winner. All tables are generated from a vitest --outputJson report by bench/benchmarks-to-markdown.mjs, run via pnpm bench:tables. The streaming suite now registers the same full event surface as the tokenize suite (comment and PI noops added): @tuananh/sax-parser's collector slows roughly a third on EPG once those listeners are registered, verified to be independent of the events occurring, so unequal surfaces made the suites disagree about the same parsers. fast-xml-parser is dropped from the streaming suite since it cannot stream. Also adds an opt-in large-document benchmark (pnpm bench:large): seeded faker-generated product catalogues at 10/50/250 MB, gitignored and generated on demand, delivered in 64 KB chunks with the same full event surface. The 10 MB suite is part of the standard bench:tables run and published as its own section. Hand-written speedup-range prose is removed, the per-fixture tables carry exact ratios. Remaining prose verified against the report.
evoactivity
added
enhancement
Each table sits between HTML comment markers (invisible on GitHub) keyed by section and fixture. pnpm bench:tables now rewrites the tables in the document directly instead of printing them for manual pasting; stdout mode remains when --write is not passed. Unknown marker keys error, and rendered tables without a marker warn so a new fixture's marker gets placed deliberately.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Restructures BENCHMARKS.md to a table per fixture, with every table produced by a generator script and spliced into the document in place between HTML comment markers (prose stays hand-written), plus a new large-document benchmark.
Table generator (
pnpm bench:tables)vitest bench --outputJsonfeedsbench/benchmarks-to-markdown.mjs. Each section opens with an overview ranking libraries by the geometric mean of op/s across its fixtures (arithmetic would be dominated by the smallest fixture), with a vs-fastest column and a dominance rule so a library that cannot parse a fixture (tXml on RSS) is never rewarded for the gap. Then one table per fixture, sorted by op/s, with each library's exact speedup relative to the fixture winner — the same ratios vitest prints.Hand-written "X-Yx faster" prose is removed; the per-fixture tables carry exact ratios now. All remaining prose was machine-verified against the report.
Streaming suite: full event surface
The tokenize suite registers 6 event types, the streaming suite registered 4. @tuananh/sax-parser's collector slows ~35% on EPG the moment comment/cdata/PI listeners are registered — verified independent of the events occurring (stripping all 10 such events from EPG changes nothing). That mismatch made the two suites disagree about the same parsers. Both suites now register the identical full surface, and the note documents the behaviour.
fast-xml-parser is dropped from the streaming suite — it cannot stream, so the reference row brought no value.
Large-document benchmark (
pnpm bench:large)Seeded faker-generated product catalogues (real-world shape: mixed attribute density, comments, CDATA, entity-escaped text) at 10/50/250 MB, generated on demand into gitignored
test/fixtures/large/bybench/generate-large-fixture.mjs. Generation is seeded, so every machine benchmarks byte-identical documents; 250 MB generates in seconds via pooled faker values. Delivered in 64 KB chunks (Node's stream default). The 10 MB suite runs as part ofbench:tablesand is published as its own section:New devDependency:
@faker-js/faker.