ParseAll URL detection dies after the first script and misses mid-tag attributes - #497
Merged
Conversation
The script automaton state (inscript_state_pos) is reset when a script or style element is entered, but never when it exits. The dirty-parser character tracker (parseall_lastc) only advances while that state is INSCRIPT_START, without checking inscript, so whatever state the automaton holds at </script> freezes for all following HTML. The '/' of </script> itself is fed to the automaton before the exit branch runs, so even a clean script deterministically parks it in INSCRIPT_SLASH: URL detection in unknown attributes (data-*, content) goes dead after the first script on the page (#201, #203). A </script> inside a JS string or comment freezes a quote state with parseall_lastc stuck on '=', turning stray quoted tokens into phantom fetches instead. Reset the state at the three live exit sites, mirroring the entry resets. Regression test proven to fail on the unfixed parser. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
xroche
force-pushed
the
fix-201-203-parser-freeze
branch
2 times, most recently
from
July 5, 2026 18:27
f1368bc to
50c24c7
Compare
Even with the automaton unfrozen, a quoted string only qualified when its next non-space character was one of ),;>/+ or end-of-line: in HTML terms, only the last attribute of a tag (or line) could be detected. The mid-tag attributes of #201 (data-gifsrc="x.gif" data-poster=...) were structurally unreachable, making detection depend on attribute order and source formatting. In a tag outside any script, a quoted value ends at its closing quote, so waive the follower requirement there. The arm resolves the owning attribute name itself and declines for no-detect/xmlns names and non-attribute quotes: the intag_startattr lookup is unreliable mid-tag (repointed at every in-tag whitespace, stale on glued attributes), and adversarial review showed a plain intag arm fetching alt/xmlns values that master suppressed. Script and event-handler contexts keep the strict gate. Tests pin each automaton-exit reset (handler fixtures freeze lastc on ';' so they stay live), the nodetect/xmlns bypass shapes, the frozen- quote phantom-fetch mode, and the '='-resolution decoys. Runtime audits in httrack-works/issue-201-203/. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Xavier Roche <roche@httrack.com>
xroche
force-pushed
the
fix-201-203-parser-freeze
branch
from
July 5, 2026 21:24
50c24c7 to
7eacbba
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With ParseAll on, URL detection in unknown attributes (
data-*,content) died for the rest of the page after the first<script>or<style>: the automaton state is reset on script entry but never on exit, and the dirty parser's character tracker reads it without checkinginscript. Even a clean</script>freezes it (its/is fed before the exit branch runs), so this hit nearly every real page; a</script>inside a JS string freezes a quote state instead, and later stray quoted tokens get fetched as phantom URLs. The first commit resets the state at the three script-exit sites.The second commit fixes the other half: a quoted value only qualified when followed by
),;>/+or end-of-line, so mid-tag attributes (data-gifsrc="x.gif" data-poster="y.jpg") were undetectable. In a tag outside scripts a value ends at its closing quote, so the follower requirement is waived there, but only for a resolvable attribute name off the no-detect/xmlns lists: review probes showed a plain waiver leakingalt/xmlnsvalues that master suppressed. Adversarial runtime audits are in httrack-works/issue-201-203; each regression test fails on the binary without its fix. Extensionless values stay out of aggressive-parser scope by design.Closes #201, Closes #203.