Contributing

Pages Parsec parses differently from a browser, selectors it gets wrong, things Lexbor does that Parsec can't, fixes, faster loops: all welcome. This page covers how the code is organised for contributors, how to build and test, and how a change gets in. The rules themselves are in CONTRIBUTING.md; this is the longer explanation.

On this page: The sync with Netweir · Building and testing · Fuzzing · Writing a change · The CLA · The wiki · Being decent

Questions go to Discussions and bugs to Issues.

Caution

Security problems go through SECURITY.md, never a public issue.

For anything bigger than a small fix, open an issue first so we can agree on the shape. Issues labelled good first issue are small and self-contained, and you'll get help through them.

The sync with Netweir

Parsec's parser has two homes. It's developed in Netweir, a scraping library, as crates/netweir-html, and copied into this repository as crates/parsec. That way one parser gets the testing of both projects, and a fix in either reaches both.

What's synced

These paths in crates/parsec are copies of Netweir's crates/netweir-html:

• src
• tests/css.rs, tests/tokenizer.rs, tests/tree.rs
• examples
• scripts
• fuzz/fuzz_targets

scripts/sync-from-netweir.sh does the copying, renaming as it goes (netweir_html becomes parsec, netweir-html becomes parsec-html), and records the Netweir commit it copied from in NETWEIR_COMMIT at the repository root.

Everything else is Parsec's own and changes here directly: the manifests (Cargo.toml files), the test data (the html5lib-tests submodule and the pinned WPT files in tests/tree-construction), the CI workflows, the documents at the root, and every crate other than crates/parsec.

How a parser change gets in

Important

Parser changes start in Netweir, never here. Edit a synced path in this repository alone and the drift check turns red.

flowchart LR
    pr["Netweir PR<br/>crates/netweir-html, with its test"] -->|"merged"| sync["sync-from-netweir.sh<br/>run from a Netweir checkout"]
    sync --> ppr["Parsec PR<br/>copied files and NETWEIR_COMMIT"]
    ppr --> drift{"Drift check<br/>matches Netweir?"}
    drift -->|"yes"| green["Drift check passes"]
    drift -->|"no"| red["Red: crates/parsec<br/>differs from Netweir"]

Loading

1. Make it in Netweir, in crates/netweir-html, as a pull request there, with its test.

2. Once it's merged, open a pull request here that runs the sync from a Netweir checkout:

   scripts/sync-from-netweir.sh ../netweir

   That copies the files and moves NETWEIR_COMMIT to the commit you synced from.

3. CI's drift job checks out Netweir at the commit in NETWEIR_COMMIT and runs scripts/sync-from-netweir.sh --check against it. If crates/parsec doesn't match, it fails. That's what an edit made here alone looks like, so don't edit the synced paths in this repository; it'll be red.

The issues for parser work say this too, and if you'd like to take one, say so on the issue and you'll be pointed at the right place in both repositories.

Building and testing

You need Rust 1.88 or newer, and nothing else.

git clone --recurse-submodules https://github.com/parsec-html/parsec
cd parsec
cargo test

If you cloned without --recurse-submodules, the tokenizer tests can't find html5lib-tests:

git submodule update --init

Conformance explains the suites, what the counts mean, and the TREE_TEST and TREE_BRIEF variables for chasing a failing tree-construction case.

What CI runs

Run these before you push and you'll save a round trip:

cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo deny check

CI runs formatting and clippy with warnings as errors, cargo deny for licences, bans and sources (and separately for advisories, which shows red without blocking), the drift check, and cargo test --workspace on Ubuntu, macOS and Windows. A CLA check and an OpenSSF Scorecard run as well.

Fuzzing

The fuzz target, crates/parsec/fuzz/fuzz_targets/parse.rs, takes arbitrary bytes. The first byte picks scripting on or off and one of 13 fragment contexts (body, div, table, tr, select, template, textarea, script and html in HTML; svg and foreignObject in SVG; math and annotation-xml in MathML); the rest is decoded lossily as UTF-8 and parsed both as a document and as a fragment. After each parse it walks the tree and asserts that every node is reached once and that every child's parent link points back.

It builds on nightly Rust with cargo-fuzz. From crates/parsec:

cargo install cargo-fuzz
cargo +nightly fuzz run parse

CI runs it every night for five minutes with inputs of up to 4,096 bytes. Locally, longer inputs find different things:

cargo +nightly fuzz run parse -- -max_len=65536 -max_total_time=3600

A crash goes in a private report, as SECURITY.md says, with the input that triggers it.

Writing a change

• A bug fix comes with the test that would have caught it. A new feature comes with tests, and with an update to the wiki page that documents it.
• A new hostile input that's slow goes into examples/hostile.rs (in Netweir) with the fix that makes it linear.
• Commit subjects take a conventional prefix, describe the problem rather than the action, and stay within 72 characters: fix(tree): a table inside a template lost its foster parent. The body, wrapped at 72 columns, says what was wrong and what changed.
• One change per pull request. The template asks what it fixes and how you tested it.

The CLA

Parsec is Apache-2.0. Its parser is shared with Netweir, which is AGPL-3.0 and also sold under a commercial licence, so code that lands here may go there too. That only works if the maintainer can license each contribution under those terms as well, so before your first pull request is merged you'll be asked to agree to the CLA. You keep the copyright in your work.

A check on your pull request asks you to post one comment, once:

I have read the CLA and I agree to it.

The wiki

This wiki is Parsec's documentation. Anyone with a GitHub account can suggest a fix to a page; for anything bigger, open an issue first. When the code changes, the page that documents it changes in the same piece of work.

Being decent

Everyone taking part agrees to the Code of Conduct.

See also

• Architecture, for a tour of the code before you change it.
• Conformance, for the test suites and how to chase a failing case.
• Roadmap, for what's planned and in what order.