fix(gates 50/57/60/62): the exemption list was the defect in all four - #379
Merged
Conversation
Each of these four gates was measured against a repo-shaped fixture with the prediction written first. In every case the detection logic was sound and the vocabulary or exemption list was what made the gate quiet. gate-60 icon-vocabulary — half the gate was unreachable code. All ten entries in CONCEPT_LABELS resolved to Tier A concepts while the vocabulary ships 140 Tier B ones, so the Tier B branch could never execute. Across 14 repos and 417 manifests it had produced zero warnings, ever. The label table is now derived from the vocabulary itself, so every concept is reachable by its own name and adding a concept makes it enforceable. Three Tier A concepts were dead too. Also narrows the missing-toolchain skip to the manifests that actually need the library, so a clean all-vocabulary app stops reporting SKIPPED. gate-57 orphaned-write-capability — the verb list is the whole detector, and it saw one of four orphaned write methods in a controlled probe. Adds the ordinary write verbs, each measured across twelve repos before inclusion and each finding read. apply/process/register were measured and rejected. One measured false positive (a pure schedule() accessor) drove a prefix-only rule for noun-shaped verbs. gate-50 security-config-fail-mode — accepted app-id shapes let it pass over sixteen unguarded reads of the opencatalogi#86 keys, in opencatalogi, because the app id is spelled $this->appName. The noise that blocked this widening before turns out to be a different variable entirely: the array-literal value position, where no guard can be written. Demoted reads are recorded in a notes sidecar rather than silenced. gate-62 store-plane — resolves an undeclared divergence with gate-60 over whether a type:"caption" node is a menu entry. Decided against the renderer: CnAppNav passes a caption only its label, so its icon draws nothing and gate 60 was right. Scoped to the icon and route rules; gate 63 keeps walking captions because labels are rendered. Adds the first acceptance suite gate-60 has ever had, plus new arms for 57, 62 and the moved gate-50 boundary. Every new arm was verified to fail against the pre-fix checkers.
rubenvdlinde
added a commit
that referenced
this pull request
Aug 12, 2026
…ented a count Two repairs, both from the same rule: a crashed checker must never read as an empty scope, and an empty scope must never read as a pass. gate-61 — the first draft of the delta-base change skipped the checker invocation entirely when no base was resolved and short-circuited to the empty-scope status. Nothing ran, so nothing could crash, and test_gate_crashed_checker_is_not_a_finding.sh caught it immediately: with a python3 that cannot run, gate-61 reported NOT APPLICABLE where it must report SKIPPED (wiring). That is the same regression #364's first draft made in gate-16, caught by the same suite, and the repair is the same one: always run the checker, evaluate wiring first and scope second. With no base the helper is now invoked in its whole-tree mode purely to learn whether it can run, and its findings are discarded — sweeping on a baseless run was tried before and reverted because the builder runs that way. gate-24 — the parity wrapper is app-owned and declares no terminal marker, so a wrapper that exited non-zero having written nothing parseable was reported as one parity violation: a fabricated number with a plausible message and nothing behind it, the same shape as gates 22, 25 and 28. The verdict deliberately stays FAIL, because nothing here can tell a crash from a differently-phrased finding and guessing wiring would turn a real parity failure into a green hole. It simply stops claiming a count nobody measured. gate-61 carries the same clamp idiom and it is left alone with the reason written down: its wiring branch sits immediately above, so a crash cannot reach it, and zero counted lines with a non-zero exit is format drift rather than a dead checker. Two suites updated, neither weakened. The crashed-checker suite now accepts gate-17's NOT APPLICABLE on its E2BIG arm only when the terminal count marker proves the interpreter ran over the oversized scope list, which is a stronger assertion than the two it sits beside; its verdict regex also gained the multi-word form that made NOT APPLICABLE parse as NOT. The route-auth suite's gate-14 expectation moves from PASS to NOT APPLICABLE for the same reason its gate-5 neighbour moved in August: a pass here is scoping, not absence, and that reclassification was only half done. Refs: .github#374, #330, #364, #379
rubenvdlinde
added a commit
that referenced
this pull request
Aug 12, 2026
* feat(gates)!: scan the whole tree by default — ADR-020 superseded The gates now scan the ENTIRE tracked codebase. Diff scoping becomes opt-in via --scope-to-diff / --diff / HYDRA_GATE_SCOPE=diff. Ruben's rationale: when a gate changes or is added, the next push to beta should fail unless the old code is brought up to the new standard, so gate updates travel with releases instead of applying only to code written after them. Recorded in hydra-gates/ADR-020-SUPERSEDED.md, including what it costs: the first development-to-beta run after this lands surfaces the fleet backlog at once, and the last wide-scope measurement was roughly 3,900 findings. The scope is now two independent, named inputs rather than one. File scope controls which files the state gates open and defaults to full; the delta base is resolved separately and is what the five delta gates compare against. Gates 16, 29, 47, 48 and 61 are keyed on whether a base was resolved, not on the file scope, so a pull request keeps every delta gate and gains whole-tree state coverage. Without a base they report NOT APPLICABLE by name, never PASS. An unresolvable base stays fatal on a diff-scoped run and is non-fatal at full scope, where it costs five gates rather than all sixty-four. BASE_REF no longer carries a hardcoded origin/development default, which was the shared root cause of the gate-16 and gate-61 empty-scope defects. Eighteen gates could report PASS over a scope they never opened. Full scope removes the empty set in the common case but leaves the fall-through latent, so it is fixed at the fall-through through one shared helper: gates 14, 17, 18, 20, 21, 22, 34 through 44 and 52. Gates 14 and 20 were found by sweeping the table and are not in the issue's list of sixteen. Gate 40's inline note arguing against this was false on both of its claims and is deleted with the fix; gate 52's uncomputed ratchet half and gate 53's uncomputed orphan-promotion half now say so on stdout instead of hiding behind a PASS. APP_DIR is absolutised before the cd and the resolved absolute path is printed. Gate 17 was the only gate handing APP_DIR to its checker afterwards, so a relative app-dir made the scan root resolve a second time against the app dir itself: same tree, absolute path FAIL 1, relative path PASS. CI was never exposed because bin/hydra-gates absolutises; the documented human invocation was. The acceptance suite now enforces the empty-scope property across the package instead of for seven named gates, and carries a relative-path arm the standing driver is structurally unable to express because it always builds absolute paths. Both new arms were proven able to fail by deliberate breakage. Helper suites 74 passed / 2 quarantined / 1 failed before and after, the failure being the pre-existing unresolvable-ajv needs-external skip that fails closed locally and is green in CI. Entry-point tests 61 passed / 0 failed before, 69 passed / 0 failed after. Refs: .github#374, #361, #347, #364, #371 * docs(gates): the runner header described the old default The Options block still said --scope-to-diff was the mode and named origin/development as the base default. Both are gone: full scope is the default, and BASE_REF starts empty because that hardcoded default is the shared root cause of .github#347 and #361. * fix(gates): a crashed gate-61 read as an empty scope, and gate-24 invented a count Two repairs, both from the same rule: a crashed checker must never read as an empty scope, and an empty scope must never read as a pass. gate-61 — the first draft of the delta-base change skipped the checker invocation entirely when no base was resolved and short-circuited to the empty-scope status. Nothing ran, so nothing could crash, and test_gate_crashed_checker_is_not_a_finding.sh caught it immediately: with a python3 that cannot run, gate-61 reported NOT APPLICABLE where it must report SKIPPED (wiring). That is the same regression #364's first draft made in gate-16, caught by the same suite, and the repair is the same one: always run the checker, evaluate wiring first and scope second. With no base the helper is now invoked in its whole-tree mode purely to learn whether it can run, and its findings are discarded — sweeping on a baseless run was tried before and reverted because the builder runs that way. gate-24 — the parity wrapper is app-owned and declares no terminal marker, so a wrapper that exited non-zero having written nothing parseable was reported as one parity violation: a fabricated number with a plausible message and nothing behind it, the same shape as gates 22, 25 and 28. The verdict deliberately stays FAIL, because nothing here can tell a crash from a differently-phrased finding and guessing wiring would turn a real parity failure into a green hole. It simply stops claiming a count nobody measured. gate-61 carries the same clamp idiom and it is left alone with the reason written down: its wiring branch sits immediately above, so a crash cannot reach it, and zero counted lines with a non-zero exit is format drift rather than a dead checker. Two suites updated, neither weakened. The crashed-checker suite now accepts gate-17's NOT APPLICABLE on its E2BIG arm only when the terminal count marker proves the interpreter ran over the oversized scope list, which is a stronger assertion than the two it sits beside; its verdict regex also gained the multi-word form that made NOT APPLICABLE parse as NOT. The route-auth suite's gate-14 expectation moves from PASS to NOT APPLICABLE for the same reason its gate-5 neighbour moved in August: a pass here is scoping, not absence, and that reclassification was only half done. Refs: .github#374, #330, #364, #379 * fix(gate-61): the scope-matrix invariant read the FILE scope, not whether a diff existed The gate-agnostic property — a NOT APPLICABLE may not blame a diff on a run that computed none — is exactly right and stays. Its TRIGGER was wrong: it keyed on --full, when the property is about whether a diff was COMPUTED. Those are two independent inputs now, so the suite was reading the wrong one. That is the same conflation .github#347 was, one layer up, inside the test written to catch it. It fired on this branch and it fired correctly. Arm 3 now removes the BASE rather than the file scope, so every assertion it already carried holds verbatim. Arm 3b covers what used to be unreachable and is now what every PR in the fleet runs: full file scope with a real base. There, naming the diff is a true statement, and what has to hold instead is the falsifiability condition whose absence made #347 unreadable for weeks — a reason may name a diff only on a run that PRINTED the base it diffed against, so a reader can check the claim rather than trust it. Asserted from both sides. The advisory whole-tree sweep now runs in BOTH empty-scope branches. It ran only on the no-base path, because that used to be the only way to reach a state where the size of the unread backlog mattered. Full scope is the default now, so "a base resolved and excluded every registration" is the common case, and the ambiguity the sweep was added to remove had simply moved to the path everyone runs: 0 of 1 and 0 of 45 still printed identically there. The whole point of the reversal is that inherited debt stops being invisible, and leaving this gate's backlog unstated on the common path would have exempted it from exactly that. Refs: .github#374, #347 * ci(gates): the unresolvable-base check asserted the contract this PR supersedes The required "Unresolvable base ref must exit 99 with no green" step encoded ADR-020's rule as the ONLY acceptable behaviour, so this PR could not merge: under the new default an unresolvable base no longer aborts, it falls back to full scope. The old assertion is kept BYTE-FOR-BYTE and simply run in the mode it was always about (--scope-to-diff): with an explicit diff request and no usable base, the runner must still exit 99, print NOTHING WAS CHECKED, and emit no gate lines. Every clause of it still holds. A second step covers the new default: the run must say the base was unresolvable, must still emit gate lines, and none of gates 16/29/47/48/61 may print PASS over a scope that was never read. Positive-controlled against this runner's real captured output four ways — injecting "[gate-16] ... PASS" fails it, deleting a gate line fails it, and the diff-mode arm fails if the 99/NOTHING WAS CHECKED behaviour regresses.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The pattern
Four gates, one shape: the detection logic was sound in every case and the vocabulary or exemption list was what made the gate quiet. Each defect below was reproduced on a repo-shaped fixture with the prediction written down first, and each fix was measured on real repos against
origin/mainat57bcb2b.Note on reading the numbers: they are written as prose and plain tables on purpose. This PR's description is echoed into the gates log via the workflow
env:block, so pasting verdict-shaped lines here would plant them inside the artefact meant to prove the fix.gate-60 icon-vocabulary — half the gate was unreachable code
All ten entries in
CONCEPT_LABELSresolved to Tier A concepts, while the vocabulary ships 140 Tier B ones. The Tier B warning branch could not execute for any input at all. Across fourteen repos and 417 manifests it had produced zero warnings, ever, so a green verdict said nothing about 140 of 153 concepts. Three Tier A concepts were dead the same way, and those are MUSTs.The label table is now derived from the vocabulary itself, so a concept is reachable by its own name and adding one to the schema makes it enforceable. The hand-written map survives as a synonym layer for Dutch spellings and abbreviations. Fleet effect: warnings nought to sixty-two, and one new failure which is a true positive (a schema labelled Audit-trail drawing the wrong glyph).
Also narrows the missing-toolchain skip. Only a non-vocabulary icon name actually needs the library, so an all-vocabulary app now gets a real verdict with no
npm ci— which is the part of "this gate went unrun" that lives in the checker rather than the workflow.gate-57 orphaned-write-capability — the verb list is the whole detector
Everything else in that module only removes findings, so a missing verb is not a weaker check but no check. Measured on a controlled probe: four zero-caller write methods on one service, one reported.
Every candidate verb was measured across twelve repos before inclusion and every finding it produced was read. Three were measured and rejected —
apply,processandregister— with their costs recorded in the code. One measured false positive, a pureschedule()accessor returning a depreciation table, drove a prefix-only rule for noun-shaped verbs. Fleet effect: forty-one to seventy-five, matching the prediction exactly.gate-50 security-config-fail-mode — blind in the app it was built for
The accepted app-id shapes let it pass over sixteen unguarded reads of the
opencatalogi#86keys, in opencatalogi, because the app id is spelled$this->appNamerather than a literal.The earlier attempt at this widening was backed out for noise, and the code carried a comment blaming the app-id shape. That diagnosis was wrong. The noise comes from the read sitting in an array-literal value position, where none of the six remedies this gate recognises can be written — no local variable, no statement slot. The two populations separate perfectly on that one syntactic fact: every one of the forty-eight noisy new findings is in that position and none of the sixteen real ones is.
So the app id is widened and the position is exempted instead. Demoted reads are written to a notes sidecar with file, line and key, so "the gate saw it and demoted it" stays checkable rather than becoming a silence. This does stop twenty-one currently-failing findings from failing; all twenty-one were read and all sit inside single
return [...]config-assembly literals.gate-62 store-plane — an undeclared divergence, now decided
gate-60 and gate-62 disagreed about whether a
type: "caption"node is a menu entry. On one fixture, gate-60 found nothing and gate-62 failed twice.Decided against the renderer rather than by aligning whichever was easier to change.
CnAppNav.vuepasses a caption only its label and a test id — no icon slot, no route, no children loop — its docblock states that captions ignore icon, route and children, and the manifest schema says the same independently. Every rule incheck_storeis a claim about a rendered icon or a resolved route, so gate-60 was right. gate-62 is aligned to it, and gate-60 now warns about the dead keys so the information is not merely dropped.Scoped deliberately: the decision covers icon and route rules only. A caption's label is rendered, so gate-63 keeps walking captions and there is an arm pinning that.
Live exposure is zero — a positive-controlled census of 145 manifests found one caption node and none carrying a dead key. This corrects a latent contradiction, not a live count.
Acceptance arms
test_check_icon_vocabulary.pyis new — gate-60 had no test suite at all — and is auto-discovered byrun-helper-suites.sh. New classes were also added to the gate-57 and gate-62 suites, and the gate-50 boundary arms intest_gate_45_to_55_acceptance.shwere rewritten with the measurement in the comment, which is what the old arm's own comment asked for.Every new arm was verified able to fail by running it against the pristine
57bcb2bcheckers: seven of nineteen red for gate-60, four for gate-57, two for gate-62, and exactly the arms that should be. The anti-widening and regression arms stay green on both sides. The full package suite and the 45–55 acceptance suite are green on the final state.Reviewer notes
run-hydra-gates.sh. gate-50's checker is a heredoc inside that file, which another agent is editing this session. I touched only the gate-50 heredoc — no scope resolution, no verdict wiring, no other gate. Please resolve in favour of both changes rather than taking a side.scripts/lib/like every other gate, so it can be unit-tested directly.fleet-board/findings/gates-exemption-fix.md.🤖 Generated with Claude Code