The a11y gate family scans src/**/*.vue only — a PHP-template app (nldesign) gets a green over an EMPTY SCOPE, and the PASS is indistinguishable from a measured zero

[rubenvdlinde]

Summary

gate-40 (form-label-association) and its neighbours in the accessibility family select their inputs with find src -name '*.vue'. nldesign has src/ but zero .vue files — it is a settings-only app whose entire UI is templates/settings/admin.php plus vanilla js/admin.js. The gate arm runs, the in-scope set is empty, and it reports:

[gate-40] form-label-association: PASS

That PASS describes nothing. It is not wrong at the gate level — the code even documents the choice, correctly, so the applicability table does not drift. But in the repo that owns the fleet's WCAG AA posture, the accessibility gates measure zero bytes, and nothing in the output says so.

Positive control

A green over an empty scope and a green over a clean scope are the same string, so I planted a defect:

<input type="text" id="gate40-probe-unlabelled" name="probe">

directly into templates/settings/admin.php — an <input> with no aria-label, no <label for>, no wrapping <label>, and a type outside the exempt set. Textbook gate-40 true positive.

mutation applied: unlabelled <input> planted
[gate-32] semantic-controls: PASS
[gate-40] form-label-association: PASS
[gate-41] html-lang: PASS

Unmoved. The gate cannot see this repo's markup, and there is no signal that distinguishes that from "this repo is clean".

What the unscanned markup actually contains

Audited by hand, since nothing else does. templates/settings/admin.php, 18 form controls:

• 15 exposed controls, all correctly associated via <label for> — nldesign-token-set-select, nldesign-hide-slogan, nldesign-show-menu-labels, nldesign-custom-css-enabled, nldesign-email-footer-*, nldesign-marianne-enabled, nldesign-upstream-freshness-toggle, and the rest.
• 3 unlabelled, all <input type="file" ... style="display:none"> (nldesign-upload-input, nldesign-font-input, nldesign-config-bundle-input). Each is triggered programmatically by a visible button. display:none removes an element from the accessibility tree entirely, so there is no accessible name to get wrong and no keyboard or AT path to it. SC 1.3.1 / 3.3.2 do not apply. Genuine non-findings.

So the repo is clean. That is a good outcome and it was not what the green told anyone — the green would have read identically over 18 unlabelled controls.

Why this matters beyond one repo

The fleet-wide claim "gate-40 measured 58% false positives" is computed over repos the gate can see. A repo it cannot see contributes a PASS to the same denominator. Any fleet a11y roll-up that counts nldesign as green is counting a repo that was never measured.

Suggested fix

Two parts, and the first matters more than the second:

1. Say when the scope is empty. Something like PASS (0 files in scope) on the [gate-N] line, for every [ -d src ] gate whose in-scope set is empty. This does not change any verdict or the applicability table — it changes an unfalsifiable string into a falsifiable one, and it is the same principle already applied in fix(gates): 16 gates reported PASS when their helper never ran — 2 are authorization gates #147 (a missing helper must not report PASS) and in gate-22/gate-53 (refuse to run fail-open rather than pass weakly).

2. Extend the input set to server-rendered markup — templates/**/*.php at minimum, for the label-association and semantic-controls gates. check_form_labels.py already parses HTML-ish markup and already excludes <script>/<style>; PHP templates need the same treatment plus <?php ?> stripping. nldesign is the only fleet app that is template-only today, but every app ships templates/settings/admin.php, and none of them is scanned.

Part 1 alone would have made this discoverable in seconds. Part 2 is what makes the gate true.

Repro

git clone https://github.com/ConductionNL/nldesign && cd nldesign && git checkout development
find src -name '*.vue' | wc -l          # 0
bash /path/to/hydra-gates/scripts/run-hydra-gates.sh   # full repo, NOT --scope-to-diff
# [gate-40] form-label-association: PASS
# ... now plant an unlabelled <input> in templates/settings/admin.php and run again: still PASS

Related: #214 (gate-38 scopes templates/settings/*.php and gets it wrong in the other direction — it does look at that file, and reports a skip link that Nextcloud core already emits).

[rubenvdlinde]

The definitive experiment: 10 planted defects, 13 gates, 0 fired

My earlier probe used one unlabelled <input>. That was too narrow to show the size of this. I planted one textbook true positive per gate into templates/settings/admin.php — nldesign's only markup file — and re-ran the full suite:

<img src="x.png">                                        <!-- gate-31 img-alt -->
<div onclick="doThing()">Save</div>                      <!-- gate-32 semantic-controls -->
<img src="y.png" alt="">                                 <!-- gate-35 img-alt-empty-only -->
<a href="#x" tabindex="5">Jump</a>                       <!-- gate-36 tabindex-positive -->
<button aria-hidden="true">Hidden but focusable</button> <!-- gate-37 aria-hidden-focusable -->
<button id="probe-nameless"></button>                    <!-- gate-39 button-name -->
<input type="text" id="probe-unlabelled" name="probe">   <!-- gate-40 form-label-association -->
<a href="/docs">click here</a>                           <!-- gate-42 link-text-quality -->
<table><tr><td>a</td></tr></table>                       <!-- gate-43 table-headers -->
<input type="password" id="probe-pw" name="pw">          <!-- gate-44 autocomplete-attr -->

Result:

[gate-31] img-alt: PASS                    [gate-39] button-name: PASS
[gate-32] semantic-controls: PASS          [gate-40] form-label-association: PASS
[gate-34] window-confirm: PASS             [gate-41] html-lang: PASS
[gate-35] img-alt-empty-only: PASS         [gate-42] link-text-quality: PASS
[gate-36] tabindex-positive: PASS          [gate-43] table-headers: PASS
[gate-37] aria-hidden-focusable: PASS      [gate-44] autocomplete-attr: PASS
                                           [gate-45] prefers-reduced-motion: PASS

Not one fired.

Why — the input selector, gate by gate

Read out of the runner:

gate	input
31, 32, 34, 35, 37, 39, 40, 42, 43, 44, 45	find src -name '*.vue'
38 skip-link	find templates/settings -name '*.php'
41 html-lang	find templates appinfo/templates -name '*.php'

Eleven of the thirteen read src/**/*.vue, and nldesign has zero .vue files. It is a settings-only app: the entire UI is templates/settings/admin.php plus vanilla js/admin.js. The eleven are not lenient here, they are blind — and they say PASS.

Only gate-38 and gate-41 read this repo's markup at all, and gate-38 is the one that is wrong (#214 here, #216 for the mount-point framing).

What this does to fleet a11y numbers

The "gate-40 measured 58% false positives" figure is computed over repos the gate can see. A repo it cannot see contributes a PASS to the same denominator, indistinguishable from a clean one. Any fleet a11y roll-up counting nldesign as green is counting a repo that was never measured — and nldesign is the app that owns the fleet's theming and WCAG AA posture.

The repo is actually clean, which is the point

Hand-audited templates/settings/admin.php: 18 form controls, 15 exposed and all correctly associated via <label for>; the other 3 are <input type="file" style="display:none">, triggered programmatically, absent from the accessibility tree — genuine non-findings.

So the answer the gates gave happened to be right. They just did not compute it, and nothing in the output distinguishes those two states.

Reinforces the suggestion above: part 1 (PASS (0 files in scope)) is worth more than part 2. It costs nothing, changes no verdict, and turns thirteen unfalsifiable greens into thirteen falsifiable ones.