✨ feat(treebuild): retarget the parser at a custom tree#601
Merged
Conversation
Merging this PR will not alter performance
Performance Changes
Comparing Footnotes
|
4e0edaa to
8b23828
Compare
Add turbohtml.treebuild.parse_into, which runs the WHATWG tree builder and drives a caller-supplied builder object -- one create_* per node kind plus an append -- to construct the tree directly, returning whatever the builder made its document root. No navigable Node is materialized and the tree is walked once, so an index, a diff tree, or another library's nodes is populated straight from the parse. The engine parses into the arena-backed C tree, then walks it in document order with an iterative heap spine (no recursion, so a deep document cannot exhaust the C stack) driving the builder. Elements carry their namespace URI and attributes as (name, value) pairs, a template's content is appended under the template handle, and a bogus <?...> reaches a distinct create_pi, matching the SAX stream. This is html5ever's TreeSink and parse5's TreeAdapter in turbohtml shape.
Register the treebuild operation across the bench registry, the CodSpeed loader, and the PGO training corpus, so it is timed on the real 235 kB spec alongside the SAX walk and gated against regression.
Add the how-to, explanation, reference autodoc, tutorial step, and the parse5 tree-adapter migration section for the pluggable tree builder, with a changelog fragment. Widen the doc-snippet linter to allow builder example methods that ignore self and their protocol arguments.
Exercise every node kind, namespace, template flattening, the doctype and processing-instruction variants, deep nesting, and the builder error paths against the SAX walk, plus an lxml round-trip differential (omitted from the coverage gate where the oracle is absent).
8b23828 to
3587dda
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pipelines that only need a projection of a page -- a link index, a diff tree, an extraction record, another library's nodes -- pay twice with
turbohtml.parse: once for the document-sized graph ofNodewrappers, and again to walk it and copy what they want out. This closes #545 by letting the same conformant parser build straight into a structure the caller controls.turbohtml.treebuild.parse_intoruns the ordinary WHATWG tree builder and drives a caller-supplied builder object -- acreate_*method per node kind plus anappendlinking a child under its parent -- to construct the tree in one document-order pass, returning whatever the builder made its document root. No navigableNodeis created and nothing is walked a second time. 🌳 The builder returns opaque handles that turbohtml only threads back into a laterappend, so a handle can be a node in your tree, an id, orNonewhen you only want the calls. It is the shape Rust's html5ever exposes as itsTreeSinktrait and Node's parse5 as itsTreeAdapter, so a parse5 adapter ports method for method.The engine parses into the arena-backed C tree, then walks it with an iterative heap spine rather than recursion, so a deeply nested document cannot exhaust the C stack. Elements carry their namespace URI and attributes as
(name, value)pairs, a<template>'s content is appended under the template handle, and a bogus<?...>construct reaches a distinctcreate_pi, the one place the builder is finer-grained than the parsedDocument, matching the SAX stream'sProcessingInstruction. The tree construction, the walk, and the string extraction all stay in C; the builder is the single Python boundary.The operation joins the benchmark registry next to the SAX walk, gated on the 235 kB spec, and the parse5 migration guide gains a tree-adapter section alongside the how-to, explanation, reference, and tutorial coverage.