fix(gate): lex the cross-product detector's alt blocks as CODE, not raw text (#76)#5718
Merged
Conversation
…aw text (#76) Family (D) of the parser-combinator gate finds an `alt((...))` block's end by counting parentheses over RAW text. That count is a lex, and a lex that reads literals and comments as code is the ceiling class that has now bitten this repo's census scanner three times (raw strings #5704, lifetimes #5705, backslash-continued strings #5715). Here it has teeth in BOTH directions, because the gate it feeds is commit-blocking: FALSE HIT an `alt((` written inside a COMMENT opens a phantom block, which swallows the ordinary tag() bindings below it and blocks a good commit over an `alt` that does not exist. FALSE MISS a comment carrying `))` exhausts the counter's depth headroom and truncates a REAL block: the arms are never collected and a genuine cross-product alt ships past the gate built to stop it. This is string-matching parsing landing on main with a green gate. Both are witnessed and committed as tests. In-tree today the bug is live-by-luck: 9 of 2,853 alt blocks in crates/engine/src/parser have their end line computed wrong (all of them phantom blocks opened by an `alt((` inside a comment), but zero FLAG DECISIONS change. Equivalence sweep old-vs-new over every block, with every line treated as added: 0 decision changes, and the block count falls 2,853 -> 2,844, exactly the 9 phantoms and nothing else. Positive control on the sweep harness: it reports a decision change on both witnesses, so the zero is a finding rather than a dead probe. DEPENDENCY DIRECTION, deliberately narrow: family (D) now imports the census module's lexer (scripts/zone_authority_census.py). Single authority for CODE-STREAM lexing -- explicitly NOT for literal matching. The detector reads STRUCTURE (where does `alt((` open, where do its parens balance?) from the stripped code stream, and CONTENT (which literals are the arms? is there an allow-noncombinator annotation?) from the raw text. Two views, one grammar. The same commit documents why the gate's OTHER families must NOT follow. (A), (B), (E) and (F) match ON the string literal (`.contains("`, `"lit" =>`, `== "long"`, `name: "..."`); routing them through strip_noncode would delete the very quote they key on and BLIND the gate -- it would pass string-matching parser code forever while reporting green. Measured during #76: 194 real match arms go unmatched that way. Their exposure to this class is the benign mirror image, a false hit that is loud and already escape-hatched (`// allow-noncombinator:`); #76 measured 10 such lines in the whole parser scope, all doc comments describing the forbidden pattern. A comment block at the grep definitions pins this where a future "hardener" would edit. The detector's seam suite runs from the gate (section D0), ahead of the scan it protects -- the same shape as section (B0) of check-engine-authorities.sh, and for the same reason: a lexing regression here does not FAIL this gate, it silently mis-scopes it. Watched red (a deliberately broken suite fails the gate) and green. Zone and draw baselines are untouched and byte-identical; this change is gate-internal. Audit outcome for the other scanners in #76: scripts/audit-parser.py is NOT A MEMBER (it never opens a .rs file -- its only input is card-data.json), and scripts/gen-test-fixture.py is benign-directional (over-harvest is fixture bloat; a missed card is loud at test runtime). Neither is changed.
matthewevans
enabled auto-merge
July 13, 2026 03:51
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Family (D) of the parser-combinator gate finds an
alt((...))block's end bycounting parentheses over RAW text. That count is a lex, and a lex that reads
literals and comments as code is the ceiling class that has now bitten this
repo's census scanner three times (raw strings #5704, lifetimes #5705,
backslash-continued strings #5715). Here it has teeth in BOTH directions,
because the gate it feeds is commit-blocking:
FALSE HIT an
alt((written inside a COMMENT opens a phantom block, whichswallows the ordinary tag() bindings below it and blocks a good
commit over an
altthat does not exist.FALSE MISS a comment carrying
))exhausts the counter's depth headroom andtruncates a REAL block: the arms are never collected and a genuine
cross-product alt ships past the gate built to stop it. This is
string-matching parsing landing on main with a green gate.
Both are witnessed and committed as tests. In-tree today the bug is live-by-luck:
9 of 2,853 alt blocks in crates/engine/src/parser have their end line computed
wrong (all of them phantom blocks opened by an
alt((inside a comment), butzero FLAG DECISIONS change. Equivalence sweep old-vs-new over every block, with
every line treated as added: 0 decision changes, and the block count falls
2,853 -> 2,844, exactly the 9 phantoms and nothing else. Positive control on the
sweep harness: it reports a decision change on both witnesses, so the zero is a
finding rather than a dead probe.
DEPENDENCY DIRECTION, deliberately narrow: family (D) now imports the census
module's lexer (scripts/zone_authority_census.py). Single authority for
CODE-STREAM lexing -- explicitly NOT for literal matching. The detector reads
STRUCTURE (where does
alt((open, where do its parens balance?) from thestripped code stream, and CONTENT (which literals are the arms? is there an
allow-noncombinator annotation?) from the raw text. Two views, one grammar.
The same commit documents why the gate's OTHER families must NOT follow. (A),
(B), (E) and (F) match ON the string literal (
.contains(","lit" =>,== "long",name: "..."); routing them through strip_noncode would delete thevery quote they key on and BLIND the gate -- it would pass string-matching parser
code forever while reporting green. Measured during #76: 194 real match arms go
unmatched that way. Their exposure to this class is the benign mirror image, a
false hit that is loud and already escape-hatched (
// allow-noncombinator:);#76 measured 10 such lines in the whole parser scope, all doc comments describing
the forbidden pattern. A comment block at the grep definitions pins this where a
future "hardener" would edit.
The detector's seam suite runs from the gate (section D0), ahead of the scan it
protects -- the same shape as section (B0) of check-engine-authorities.sh, and
for the same reason: a lexing regression here does not FAIL this gate, it
silently mis-scopes it. Watched red (a deliberately broken suite fails the gate)
and green. Zone and draw baselines are untouched and byte-identical; this change
is gate-internal.
Audit outcome for the other scanners in #76: scripts/audit-parser.py is NOT A
MEMBER (it never opens a .rs file -- its only input is card-data.json), and
scripts/gen-test-fixture.py is benign-directional (over-harvest is fixture bloat;
a missed card is loud at test runtime). Neither is changed.