fix(gates 19, 25, 26, 51, 52, 54, 55): the scope decision has exactly one source (#416)

[rubenvdlinde]

Closes #416.

What was wrong

#378 made whole-tree the default file scope. It did not reach pull_request — the event that gates a merge.

Seven state gates take their base ref from the environment and are invoked through a shape whose else branch spawns the checker without clearing it:

if [ "${SCOPE_TO_DIFF}" = "1" ]; then
    HYDRA_GATE_BASE_REF="${BASE_REF}" python3 .../check_x.py .
else
    python3 .../check_x.py .          # <-- inherits the AMBIENT variable
fi

That if guards the explicit pass; gate-19's copy of the comment cites #242 and explains this precise hazard. It cannot guard the environment. The shared quality workflow exports the variable on pull_request and leaves it empty on push — so those seven silently diff-scoped themselves on pull requests and swept the tree only on the push that happens after the merge, where nothing can block.

The issue named gates 19, 25 and 26. Reading the runner, the same unguarded shape is at seven sites — 19, 25, 26, 51, 52, 54 and 55 — and all seven helpers read the variable from os.environ.

The control, before anything was changed

Fresh clone of ConductionNL/docudesk development at 272ec7a1 plus one commit touching only README.md — the shape of a one-file PR. Gate package at 5e73e640. Both arms full scope, base origin/development, one changed file. Only the delivery channel differs.

gate	base via the environment	base via --base
19 e2e-coverage	not applicable	fails, 396 scenarios
25 contract-coverage	not applicable	passes — it ran
26 visual-coverage	not applicable	fails, 6 page components
coverage line	56 of 65	59 of 65
exit	5 gates failed	7 gates failed

Diffing the two complete 65-gate verdict sets: exactly three lines differ, and they are 19, 25 and 26. The delta gates, which read the same resolved base, are unaffected by the channel.

The sharpest form of the defect is the runner's own preamble. Both arms print "The DELTA gates (16, 29, 47, 48, 61) judge that change set. Every other gate reads the whole tree." In the environment arm that sentence is false, and the log gives the reader no way to tell. Each of the three even states a true reason — the diff really did touch no spec file — but the scope it applied is not the scope the run announced.

The fix, and why it is not the else branches

One line in run-hydra-gates.sh, immediately after the variable is consumed into BASE_REF:

unset HYDRA_GATE_BASE_REF

The runner reads the variable, then removes it from its own environment. Every checker is spawned into an environment that does not contain it, so a checker can receive a base only when this script explicitly supplies one — which is exactly what the SCOPE_TO_DIFF branches already do.

Patching the seven else branches (the remedy the issue suggested) also works. I did not choose it because that repair has to be remembered by each gate, and it already failed six times: gate-19's author wrote the #242 comment explaining the hazard, and gates 25, 26, 51, 52, 54 and 55 were each subsequently written with the same unguarded else. Under the unset, a gate written tomorrow with the identical shape is correct by construction.

The invariant this establishes: the scope decision has exactly one source — the shell variables BASE_REF and SCOPE_TO_DIFF, resolved in one place and printed. --base X and the environment variable become two spellings of one input rather than two channels, one of them unprinted.

What it deliberately does not do: it does not stop the runner reading the variable. It stays a supported input for direct callers (the builder skill, a human at a shell); only the ambient inheritance is removed. "Stop reading it" would silently retire gates 16/29/47/48/61 for every direct caller while leaving the parity property satisfied — so the new suite asserts that distinction separately.

Post-fix, same rig, same two arms: the verdict sets are identical, both report 59 of 65, both exit 7.

Side note: scripts/lib/test_gate16_spec_coverage_scope.sh:22 already asserts in prose that "gate-19's else-branch drops HYDRA_GATE_BASE_REF and scans the whole tree". That sentence was false when it was written and is true after this change.

The acceptance arm

scripts/lib/test_gate_base_ref_delivery_channel.sh — auto-discovered by tests/run-helper-suites.sh, driving the real bin/hydra-gates.

It pins the property generically: for one tree and one base at full scope, the per-gate verdict set must be identical whichever channel delivered the base. A gate added tomorrow that reads the ambient variable is caught without anyone editing the file.

Three things keep it from being vacuous, because two arms agreeing that nothing is applicable is exactly the state the defect produced:

• a positive control runs first and refuses to grade unless the fixture's uncovered scenario is provably findable;
• gate-19 must fail in both arms, naming the fixture's own scenario — agreement alone can never satisfy the file;
• the base-source labels are checked, not reasoned about, so the two arms cannot secretly be the same channel.

The fixture's subject is a single scenario whose title is a token appearing nowhere else in the package, so a finding about any other subject cannot satisfy the assertion, and the file carries exactly one finding.

Revert-proven. With the one-line change commented out, the suite goes red, names gate-19, and prints the differing pair:

   gate | --base            | $HYDRA_GATE_BASE_REF
   -----+-------------------+---------------------
     19 | FAIL — 1          | NOT APPLICABLE

Blast radius — measured on all 18 core apps, and it is the point

These are not new findings. They are the numbers the push run has been printing all along, now arriving on the event where they can block.

Fresh clones of development, each checker run with the variable absent — which is what the runner now does. Validated against the full runner on docudesk, where the direct-checker numbers are byte-identical to the post-fix wrapper verdicts.

app	SHA	gate-19	gate-25	gate-26
decidesk	e36dbc4	983	pass	pass
docudesk	0be7cbcf	396	pass	6
doriath	85d4c7d	149	pass	3
hermiq	61b3f1a	491	5	6
larpingapp	5373b2d	42	pass	pass
launchpad	30cd0bd	128	pass	1
nldesign	62a04f1	126	pass	n/a
openbuild	f17df03	158	pass	4
opencatalogi	634876f	31	pass	1
openconnector	00ab159	332	pass	pass
openregister	c9d9696	845	66	27
pipelinq	4c5fa97	33	pass	3
portaliq	3b0b6d4	pass	pass	1
procest	bf54f3c	1142	118	78
scholiq	e45aa95	93	pass	9
shillinq	0742123	290	52	2
softwarecatalog	051aa85	266	pass	5
zaakafhandelapp	86393c4	pass	pass	n/a

Totals: gate-19 5,505 · gate-25 241 · gate-26 146. portaliq and zaakafhandelapp are already clean on gate-19 and stay clean.

Four of these independently reproduce figures reported earlier in the programme — decidesk 983, docudesk 396, openconnector 332, launchpad 128 — measured here from a different rig, a different clone and a direct checker invocation.

⚠️ gate-19 is a coverage backlog, not a defect, and its only annotation-shaped remedy is scored by the gate as positive coverage (#345). Mass-excluding would turn every cell green while covering nothing. Nobody should. Every number above is also a floor, not a ceiling: exclusion inheritance (#356) only moves scenarios into excluded.

Verification

• new suite: 12 assertions, green; red on revert, naming gate-19 and printing the differing pair
• tests/test-hydra-gates-bin.sh: 70 passed, 0 failed
• tests/run-helper-suites.sh: 79 passed, 2 quarantined (pre-existing and documented), 2 failed — both disproven as failures of this change:
  • test_gate_45_to_55_acceptance.sh refuses to run when ajv is unresolvable. Reproduced identically on a pristine clone of main at the same SHA. Re-run with NODE_PATH at an ajv install it is ALL GREEN on both trees — and that is the arm covering gates 51/52/54/55.
  • test_gate_route_registration.sh aborted mid-run under my own parallel load (two full 65-gate sweeps running concurrently). Isolated re-run on this branch: 47 passed, 0 failed; the pristine control likewise 47/0.
• test_gate_scope_matrix.sh, test_gate16_spec_coverage_scope.sh, test_gate19_coverage_credibility.sh and test_gate_acceptance_matrix.sh all pass — the fixture-coverage ratchet accepted the new bundle without an entry.
• ShellCheck 0.10.0 locally: the new suite is clean; the runner's single pre-existing finding is byte-identical to main's.

What this does not establish

• No behavioural change is demonstrated for gates 51, 52, 54 and 55. Their exposure is established by reading the code — identical unguarded else, all four helpers read the variable from os.environ — and they are in scope of the fix. But their whole-tree finding count is zero in every app measured, so scoped and swept are typographically identical there. Unproven behaviourally, not immune.
• gate-25/26 numbers for 15 of the 18 apps come from the direct checker. They were validated against the full wrapper on docudesk, openregister and procest, where the two paths agree exactly; the other fifteen are one code path.
• Nothing here is CI-side. Everything was measured locally through the real bin/hydra-gates. The first app pull request after this lands is the confirming observation and is worth looking at deliberately.