Skip to content

Conformance

James Conway edited this page Nov 5, 2026 · 3 revisions

Parsec is tested against the public suites that browsers and other parsers use: html5lib's tokenizer tests and the tree-construction tests now kept in web-platform-tests. Both run in cargo test, and CI runs cargo test --workspace on Linux, macOS and Windows for every pull request.

On this page: The results · Running them · Other tests · Coverage · What isn't run yet · Next to Lexbor

Every number on this page is what the test runners print, on main.

The results

Suite Data Passed Failed Skipped
html5lib tokenizer crates/parsec/tests/html5lib-tests/tokenizer, html5lib-tests at commit c777c40 7,045 0 4
WPT tree construction crates/parsec/tests/tree-construction, web-platform-tests at commit c119765 3,898 0 6

The runners print exactly this:

tokenizer: 7045 passed, 0 failed, 4 skipped (lone surrogates)
tree construction: 3898 passed, 0 failed, 6 skipped (need JavaScript)

What those counts count is worth spelling out, because it differs between the two.

Tokenizer: 7,045 runs

The suite has 14 .test files holding 6,823 tests. A test can list several initial states (data, PLAINTEXT, RCDATA, RAWTEXT, script data, CDATA section), and the runner runs it once per state, which makes 7,049 runs. 7,045 pass. The other 4 are skipped: they're doubleEscaped tests whose input contains a lone surrogate, which a Rust &str can't hold, so there's no way to give them to Parsec.

The runner (crates/parsec/tests/tokenizer.rs) preprocesses each input, sets the initial state and lastStartTag where the test gives them, collects tokens through a Sink, merges adjacent character tokens as the suite's README says, and compares with the expected output. It doesn't check the errors field: the tokenizer recovers from parse errors as the spec says but doesn't report them yet (#7).

Tree construction: 3,898 runs of 1,967 cases

The tree-construction tests moved from html5lib-tests to web-platform-tests in 2026. The pinned copy has 62 .dat files and 1,973 test cases, 206 of them fragment cases. Each case can say it's for scripting on (#script-on), off (#script-off), or say neither:

Cases Run with Runs
1,931 that say neither scripting off, and again with it on 3,862
28 marked #script-off scripting off 28
8 marked #script-on scripting on 8
6 marked #script-on in scripted_* files skipped 0

That's 3,898 runs, all passing. Put the other way: of 1,973 cases, the 1,967 a parser without JavaScript can run all pass, in every mode they apply to.

The 6 skipped cases are in scripted_adoption01.dat, scripted_ark.dat, scripted_foster01.dat and scripted_webkit01.dat, and they run script during the parse (document.write, DOM calls), which needs a JavaScript engine. Parsec doesn't run scripts.

The runner (crates/parsec/tests/tree.rs) parses each case with parse or, for fragment cases, parse_fragment with the given context (svg x and math x become Ns::Svg and Ns::MathMl). It writes the tree in the suite's | <tag> format, with attributes sorted, template contents under content, and namespaced attributes as xlink href, and compares that with #document. The #errors and #new-errors sections aren't checked, for the same reason as above.

Running them

git clone --recurse-submodules https://github.com/parsec-html/parsec
cd parsec
cargo test

The tokenizer tests read html5lib-tests, which is a git submodule. If you cloned without --recurse-submodules, fetch it first:

git submodule update --init

To see the counts, which the runners print to stderr, run one suite with output on:

cargo test --test tokenizer -- --nocapture
cargo test --test tree -- --nocapture

Two environment variables help when a tree-construction case fails:

  • TREE_TEST=adoption runs only the files whose names contain adoption.
  • TREE_BRIEF=1 prints one line per failure (file, scripting, fragment context and input) before the full report of the first 15.
TREE_TEST=template TREE_BRIEF=1 cargo test --test tree -- --nocapture

Moving to newer tests

The WPT files aren't a submodule; they're copied into crates/parsec/tests/tree-construction with the commit in COMMIT beside them. To move them, from crates/parsec, with gh and curl installed:

scripts/fetch-tree-tests.sh WPT_COMMIT

It replaces the directory with that commit's html/syntax/parsing/resources/*.dat files and the WPT licence. Note that crates/parsec/scripts is synced from Netweir, so a change to the script itself starts there (see Contributing).

Other tests

  • crates/parsec/tests/css.rs: 12 tests for the selector engine, covering every feature in Selectors, the selectors that must be refused, and the nesting limit.
  • 27 unit tests inside crates/parsec/src: the budget, linear memory on a known hostile input, serialization on a small stack, edits that keep strings in place, the tokenizer's fast path against its state machine, and more.

Coverage

How much of the code those tests run, from the first coverage run on main, at commit adc2dcc:

Covered Total Missed
Lines 94.31% 5,062 288
Regions 93.78% 6,177 384
Functions 94.36% 532 30
Branches 85.30% 1,476 217

The coverage workflow runs cargo llvm-cov --workspace --branch on nightly (branch coverage needs it) on every push to main. It fails if lines drop under 90% or branches under 80%, the OpenSSF Best Practices gold thresholds, so a slide shows up on the change that caused it. The full report and an lcov file are kept as the run's coverage artifact.

To get the same table locally, with cargo-llvm-cov installed:

cargo +nightly llvm-cov --workspace --branch --summary-only

Coverage in CI was added by @foothillsdigital in #61.

What isn't run yet

Suite Why not Issue
html5lib encoding/ Parsec takes &str, so there's no sniffing to test yet. #5
html5lib serializer/ Needs the serializer options those tests use. #9
Parse errors in both suites Errors aren't reported yet. #7
WPT css/selectors No runner yet. #19
WPT encoding, URL; css-parsing-tests Those modules don't exist yet. #25, #26, #28

Publishing these counts from CI as a JSON summary, so this page can be generated rather than written, is planned in #4.

Next to Lexbor

Lexbor's README says it "passes all tree construction tests" but doesn't publish a count (README). Lexbor vendors 1,851 html5lib tree-construction cases in its own format (test files) and has an opt-in runner for the WPT files that skips the scripted ones (test README). The independent JustHTML project measured selectolax, which uses Lexbor, passing 1,880 of 1,880 WPT cases without scripting fixtures, at WPT commit 4830edb (comparison).

Those numbers aren't directly comparable with Parsec's: they're from a different WPT commit and count cases once rather than once per scripting mode. By those measures, both pass everything they run. Lexbor vendors the html5lib tokenizer tests too but doesn't publish a pass rate.

See also

Clone this wiki locally