You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
make coverage-docs (the tools/coverage generator behind docs/user/coverage-matrix.md) can no longer see the wired UAT lanes. Regenerating the page today silently downgrades every UAT-derived row — uat, GPU (H100, real), nightly becomes chainsaw, simulated, per-PR, and evidence verify becomes not-yet-covered — which would falsely claim nightly UAT coverage vanished.
Cause
Commit a363ce4 (UAT reservation registry, #1274/#1559) restructured the UAT workflows:
TEST_CONFIG is now composed from ${{ inputs.… }} expressions, so tools/coverage/wiring.go's uatConfigRef regex no longer matches any literal tests/uat/<cloud>/tests/*-config.yaml string in the workflow files.
Several scanned commands moved into tests/uat/lib/phases.sh, which the wired-runner scanner does not read.
Two secondary staleness items in the same generator:
The version axis still reports main only, but the nightly lanes run with previous_n=1.
make coverage-check is opt-in (not wired into make qualify or the merge gate), so the page will read as stale/wrong until the scanner is fixed, with nothing flagging it on PRs.
Impact
The committed docs/user/coverage-matrix.md is currently correct only because contributors know not to commit regenerated output; anyone who runs make coverage-docs and commits the result regresses the page.
Teach wiring.go to resolve TEST_CONFIG through the reservation-registry inputs (or scan the composite/lib layer, including tests/uat/lib/phases.sh) instead of grepping workflow files for literal config paths.
Update the version axis to reflect previous_n.
Consider wiring make coverage-check into make qualify once the scanner is trustworthy again, so drift is caught at PR time.
Inherited from main (pre-dates #1967); not a blocker for that PR.
Summary
make coverage-docs(thetools/coveragegenerator behinddocs/user/coverage-matrix.md) can no longer see the wired UAT lanes. Regenerating the page today silently downgrades every UAT-derived row —uat, GPU (H100, real), nightlybecomeschainsaw, simulated, per-PR, andevidence verifybecomesnot-yet-covered— which would falsely claim nightly UAT coverage vanished.Cause
Commit a363ce4 (UAT reservation registry, #1274/#1559) restructured the UAT workflows:
TEST_CONFIGis now composed from${{ inputs.… }}expressions, sotools/coverage/wiring.go'suatConfigRefregex no longer matches any literaltests/uat/<cloud>/tests/*-config.yamlstring in the workflow files.tests/uat/lib/phases.sh, which the wired-runner scanner does not read.Two secondary staleness items in the same generator:
main only, but the nightly lanes run withprevious_n=1.make coverage-checkis opt-in (not wired intomake qualifyor the merge gate), so the page will read as stale/wrong until the scanner is fixed, with nothing flagging it on PRs.Impact
docs/user/coverage-matrix.mdis currently correct only because contributors know not to commit regenerated output; anyone who runsmake coverage-docsand commits the result regresses the page.Proposed fix
wiring.goto resolveTEST_CONFIGthrough the reservation-registry inputs (or scan the composite/lib layer, includingtests/uat/lib/phases.sh) instead of grepping workflow files for literal config paths.previous_n.make coverage-checkintomake qualifyonce the scanner is trustworthy again, so drift is caught at PR time.Inherited from main (pre-dates #1967); not a blocker for that PR.