test(gate-28): acceptance evidence for #178 — and the three assertions the suites were missing

[rubenvdlinde]

Cause — and an honest correction up front

#178 was already fixed. #182 (da23b92, merged 2026-08-06 07:20Z, 88 minutes after the issue was filed) decides gate-28's applicability from the diff, and it is released in v1.5.2 and v1.6.0. The issue was simply never closed.

The original mechanism, for the record: scope came from the DIFF (check_license_triangle.py::main() reads files = argv[2:]), while "subject matter exists" was judged from the REPO (lib/ present + a composer licence). A PHP-free diff handed the gate zero files, it compared nothing, and reported structural — a job failure under hydra-gates-require-full-coverage. #182 made that state na, keeping structural to mean "PHP files WERE in scope and none declared anything".

This PR contains no runtime change. run-hydra-gates.sh and check_license_triangle.py are untouched — the diff is 340 added lines across the two test files, zero deletions. What it adds is the acceptance evidence, plus three gaps in what the suites were actually asserting.

The three proofs, measured on this tree

All with the real runner, --scope-to-diff --require-full-coverage, in a fixture repo that HAS lib/ and "license": "EUPL-1.2".

(a) NOT APPLICABLE — diff = README.md + src/components/Widget.vue, zero PHP:

[gate-28] license-triangle: NOT APPLICABLE — this diff touches none of the 1 tracked
lib/**/*.php file(s) ... Diff-scoped out under ADR-020 ...
[hydra-gates] ALL 49 APPLICABLE GATES GREEN — and all 49 of them ran.        (status 0)

(b) TRUE POSITIVES STILL CAUGHT — the checker has not been widened into uselessness:

# untagged PHP in scope
[gate-28] license-triangle: SKIPPED (structural) — ... 1 lib/**/*.php file(s) WERE in
scope, but not one carried an @license or SPDX-License-Identifier declaration ...

# only-gap-is-gate-28 fixture, same flag
[hydra-gates] --require-full-coverage was set: treating incomplete coverage as failure.
                                                                             (status 98)

# genuine MISMATCH: file says AGPL-3.0-or-later, composer.json says EUPL-1.2
[gate-28] license-triangle: FAIL — 1 file(s) with @license != composer.json

(c) PASS — tagged PHP in scope: [gate-28] license-triangle: PASS, 50 of 50 applicable gates ran.

What the suites were NOT asserting

1. The suite never passed --require-full-coverage. It proved "gate-28 prints NOT APPLICABLE" and inferred "the run is not failed". Different claims — and they came apart once already, when fix(hydra-gates): a not-applicable gate is not a coverage gap — fix the accounting, then flip the flag #164 defaulted the flag TRUE in the workflow while every local suite ran without it.
2. Nothing exercised the _fail 28 branch end to end. A licence mismatch is the gate's actual purpose; only the weaker "no file carried a tag" control existed. The helper stdout -> _lt_log -> wc -l -> _fail wiring was untested — a helper printing findings to stderr would keep every unit test green and turn the gate into a no-op.
3. declared_file_count() had no test in either direction, despite being the single number the PASS-vs-structural decision rests on, and nothing pinned the stdout/stderr split it travels over.

A trap found while writing this — worth reading

The coverage-failure branch lives inside if [ "${_FAILED}" -eq 0 ]. Any other failing gate suppresses that sentence entirely, so an assertion of the form "the coverage sentence is absent" passes automatically on a fixture that trips an unrelated gate — including on a fully reverted runner.

That is not hypothetical: the first draft wrote its Vue file to src/App.vue, which gate-38 (skip-link) fails as a root component. Both full-coverage runs exited 1, not 98, and both new assertions were measuring gate-38 rather than gate-28. Fixes now baked into the fixtures and documented in-file:

• fixtures are otherwise green (src/components/Widget.vue);
• the pass case reads the positive ALL N APPLICABLE GATES GREEN sentence — a sentence that must be PRESENT cannot be satisfied by a run that died earlier;
• the control uses a composer.json with no license field rather than untagged PHP, because untagged PHP also fails gate-1 and makes the coverage branch unreachable.

Tests added

suite	before	after
test_gate_license_triangle_scope.sh	5 assertions	11
test_check_license_triangle.py	19 tests	29

Both appended to — git diff --stat is 340 insertions(+), 0 deletions.

Mutation-check — every new assertion shown capable of failing

mutation	effect
M1 empty diff scope -> structural (pre-#182)	2 new shell assertions RED, status 98 — #178 reproduced exactly
M2 structural -> na (gate stops failing)	2 new shell assertions RED
M3 _fail 28 branch disabled	2 new shell assertions RED — drift became PASS
M4 declared_file_count -> len(files)	3 new python tests RED
M5 count printed to stdout	4 new python tests RED; end-to-end a clean repo turns FAIL
M6 empty scope invents a finding + a count	3 new python tests RED
M7 declared_file_count -> 0	5 new python tests RED; PASS became structural

Each mutation was applied with Edit, measured, then reverted with git checkout --; the tree was confirmed byte-identical (git status --porcelain clean) before the next.

Statuses are reported alongside the assertions but are never the assertion itself — per #209, where gate-19 returned its finding count as an exit status and 266 findings reported as 10.

Regression

hydra-gates/tests/run-helper-suites.sh: 27 passed, 0 failed, 2 quarantined (pre-existing, unrelated).

Closes #178

🤖 Generated with Claude Code