✨ feat(clean): add XML/XHTML output to the sanitizer#598
Merged
Conversation
Merging this PR will not alter performance
Performance Changes
Comparing Footnotes
|
a6202dd to
585e62e
Compare
585e62e to
c04e7cd
Compare
Policy gained an xml flag: with it on, the sanitizer serializes the
cleaned tree as well-formed XML/XHTML instead of HTML. The allowlist walk
and the non-configurable safety baseline are untouched, so an XML-mode
policy is exactly as safe as its HTML-mode twin; only the final
serialization differs. Every kept empty element self-closes, foreign SVG
and MathML subtrees declare their namespace, and text and attribute values
follow the XML escaping rules.
The well-formedness pass is opt-in on the serializer, carried by a new
well_formed serialize option that only the sanitizer's inner_xml turns on.
On that path a comment's -- and trailing - gain a space, a character
outside XML's Char production (a C0 control, a lone surrogate,
U+FFFE/U+FFFF) is dropped, a stored xmlns the serializer already declares
is not duplicated, and an attribute whose name is not a valid XML name is
skipped, so a cleaned fragment always reparses through parse_xml. A plain
Node.serialize(Html(xml=True)) stays on the raw XML path, byte- and
instruction-identical to before, so the serialize benchmarks do not move.
The new Node.inner_xml property exposes the well-formed children
serialization the sanitizer routes through.
This clones DOMPurify's application/xhtml+xml parser mode and replaces the
brittle .replace("<br>", "<br/>") a bleach-based cleaner needs to feed a
strict XHTML consumer such as Reportlab's RML. The DOMPurify XSS corpus is
run through the XML sanitizer to prove every vector serializes to inert,
well-formed XML.
closes tox-dev#565
c04e7cd to
c082f67
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sanitizing is not unique to HTML: an XHTML dialect such as Reportlab's RML or an ePub content document rejects HTML's bare
<br>, so a bleach-based cleaner has to patch its output with a brittle.replace("<br>", "<br/>")or reparse and reserialize the whole tree a second time.Policynow takes anxmlflag that removes that step: the sanitizer serializes its cleaned tree as well-formed XML/XHTML instead of HTML, sosanitize("<p>one<br>two</p>", Policy(tags=frozenset({"p", "br"}), xml=True))returns<p>one<br/>two</p>. This closes #565 and clones DOMPurify'sPARSER_MEDIA_TYPE: 'application/xhtml+xml'.The allowlist walk and the non-configurable safety baseline are untouched, so an XML-mode policy is exactly as safe as its HTML-mode twin, and only the final serialization changes. 🔒 Because the sanitizer keeps its parse-once, serialize-once shape with no reparse round trip, there is no new opening for a mutation-XSS vector to re-enter. The well-formedness pass is opt-in on the serializer, carried by a
well_formedserialize option that only the sanitizer'sinner_xmlturns on: on that path a comment's--and trailing-gain a space, a character outside XML'sCharproduction (a C0 control, a lone surrogate,U+FFFE/U+FFFF) is dropped, a storedxmlnsthe serializer already declares for a foreign root is not duplicated, and an attribute whose name is not a valid XML name is skipped. A plainNode.serialize(Html(xml=True))stays on the raw XML path, byte- and instruction-identical to before, so the serialize benchmarks do not move. The newNode.inner_xmlproperty exposes the well-formed children serialization the sanitizer routes through.Every vector in DOMPurify's own XSS corpus (its
test/fixtures/expect.mjs, vendored as a pinned submodule) is run through the XML sanitizer to confirm each one serializes to output that is both inert and reparses throughturbohtml.parse_xml, and an lxml differential checks the same well-formedness against an independent parser. Thexmlflag is off by default, so existing callers are unaffected.