Skip to content

✨ feat(treebuild): retarget the parser at a custom tree#601

Merged
gaborbernat merged 4 commits into
tox-dev:mainfrom
gaborbernat:feat/545-tree-sink
Jul 8, 2026
Merged

✨ feat(treebuild): retarget the parser at a custom tree#601
gaborbernat merged 4 commits into
tox-dev:mainfrom
gaborbernat:feat/545-tree-sink

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

Pipelines that only need a projection of a page -- a link index, a diff tree, an extraction record, another library's nodes -- pay twice with turbohtml.parse: once for the document-sized graph of Node wrappers, and again to walk it and copy what they want out. This closes #545 by letting the same conformant parser build straight into a structure the caller controls.

turbohtml.treebuild.parse_into runs the ordinary WHATWG tree builder and drives a caller-supplied builder object -- a create_* method per node kind plus an append linking a child under its parent -- to construct the tree in one document-order pass, returning whatever the builder made its document root. No navigable Node is created and nothing is walked a second time. 🌳 The builder returns opaque handles that turbohtml only threads back into a later append, so a handle can be a node in your tree, an id, or None when you only want the calls. It is the shape Rust's html5ever exposes as its TreeSink trait and Node's parse5 as its TreeAdapter, so a parse5 adapter ports method for method.

The engine parses into the arena-backed C tree, then walks it with an iterative heap spine rather than recursion, so a deeply nested document cannot exhaust the C stack. Elements carry their namespace URI and attributes as (name, value) pairs, a <template>'s content is appended under the template handle, and a bogus <?...> construct reaches a distinct create_pi, the one place the builder is finer-grained than the parsed Document, matching the SAX stream's ProcessingInstruction. The tree construction, the walk, and the string extraction all stay in C; the builder is the single Python boundary.

The operation joins the benchmark registry next to the SAX walk, gated on the 235 kB spec, and the parse5 migration guide gains a tree-adapter section alongside the how-to, explanation, reference, and tutorial coverage.

@gaborbernat gaborbernat added the enhancement New feature or request label Jul 8, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 8, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 85 untouched benchmarks
🆕 1 new benchmark
⏩ 18 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
🆕 test_feature[treebuild] N/A 24.3 ms N/A

Comparing gaborbernat:feat/545-tree-sink (3587dda) with main (fe5325a)

Open in CodSpeed

Footnotes

  1. 18 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@gaborbernat gaborbernat force-pushed the feat/545-tree-sink branch from 4e0edaa to 8b23828 Compare July 8, 2026 03:15
Add turbohtml.treebuild.parse_into, which runs the WHATWG tree builder
and drives a caller-supplied builder object -- one create_* per node
kind plus an append -- to construct the tree directly, returning
whatever the builder made its document root. No navigable Node is
materialized and the tree is walked once, so an index, a diff tree, or
another library's nodes is populated straight from the parse.

The engine parses into the arena-backed C tree, then walks it in
document order with an iterative heap spine (no recursion, so a deep
document cannot exhaust the C stack) driving the builder. Elements carry
their namespace URI and attributes as (name, value) pairs, a template's
content is appended under the template handle, and a bogus <?...> reaches
a distinct create_pi, matching the SAX stream. This is html5ever's
TreeSink and parse5's TreeAdapter in turbohtml shape.
Register the treebuild operation across the bench registry, the CodSpeed
loader, and the PGO training corpus, so it is timed on the real 235 kB
spec alongside the SAX walk and gated against regression.
Add the how-to, explanation, reference autodoc, tutorial step, and the
parse5 tree-adapter migration section for the pluggable tree builder,
with a changelog fragment. Widen the doc-snippet linter to allow builder
example methods that ignore self and their protocol arguments.
Exercise every node kind, namespace, template flattening, the doctype
and processing-instruction variants, deep nesting, and the builder
error paths against the SAX walk, plus an lxml round-trip differential
(omitted from the coverage gate where the oracle is absent).
@gaborbernat gaborbernat force-pushed the feat/545-tree-sink branch from 8b23828 to 3587dda Compare July 8, 2026 03:53
@gaborbernat gaborbernat merged commit f73cae2 into tox-dev:main Jul 8, 2026
49 checks passed
@gaborbernat gaborbernat deleted the feat/545-tree-sink branch July 10, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parser: pluggable tree-builder (TreeSink/TreeAdapter)

1 participant