Wire gate matrix to ADR-0003 pins — kill the placeholder versions.json

[myselfsiddharth]

Context

There are two version matrices in this repo and only one of them is real.

scripts/testbed/matrix.json (B1, ADR-0003) holds the eight pinned Grafana tags with churn_role notes, Docker Hub URLs, GitHub release URLs, and access_date values.

experiments/gate-v1/versions.json (B4) holds exactly one entry:

{ "id": "pending-b1@placeholder", "site_key": "local-demo", "status": "placeholder",
  "notes": "Awaiting B1 test-bed pin. Do not invent a console version here." }

experiments/gate-v1/run-matrix.ts iterates versionsDoc.versions, so npm run gate:matrix currently walks a one-element placeholder list. Any report generated from it is a report about nothing. experiments/gate-v1/README.md lists "Wire versions.json to ADR-0003 / scripts/testbed/matrix.json pins" as an open question.

Two sources of truth for the same list will drift. Collapse them.

What to build

1. Make scripts/testbed/matrix.json the single source of truth for the version list. experiments/gate-v1/run-matrix.ts reads it directly (via the existing src/testbed/matrix.ts loader rather than a second ad-hoc JSON import).
2. Keep experiments/gate-v1/versions.json only if it carries information matrix.json genuinely should not — per-run gate settings such as site_key / task_key defaults or a version subset filter. If it holds nothing but a duplicated list, delete it and say so in the PR.
3. Add a --versions <a,b,c> flag to npm run gate:matrix to run a subset, and --versions all (the default) for the full matrix. Reject any id not present in matrix.json with an error naming the valid ids — never fall through to a placeholder.
4. Skip versions marked "status": "unavailable" in matrix.json (see the tag-verification issue), log that they were skipped, and record the skip in the run output so a later report cannot silently under-count denominators.
5. Per-version and aggregate outputs land under experiments/gate-v1/out/ — already gitignored.

Constraints

• Do not invent a version pin. If matrix.json lacks something the harness wants, that is a matrix change with an ADR-0003 amendment, not an inline default.
• Do not change the --dry-run gating in this PR. Live execution is a separate issue; this one is about the harness walking the right list.
• The existing dry-run behaviour (PASS outcomes, zero tokens, "not a gate measurement" note) must survive unchanged — just over the real eight versions instead of one placeholder.

How to test

npm run gate:matrix -- --dry-run                       # walks every available matrix version
npm run gate:matrix -- --dry-run --versions 11.0.0,12.0.0
npm run gate:matrix -- --dry-run --versions 99.0.0     # must error, listing valid ids
npm run gate:report
npm run ci

Confirm experiments/gate-v1/out/metrics.ndjson contains one run row per version walked, each with the correct testbed_version, and that npm run gate:report renders them without inventing rates. Every token count stays 0 — this is still a dry run and must be labelled as such.

Before you open the PR

• npm run ci and npm run test:canary green
• Only one version list remains in the repo, or the second one is justified in the PR body
• Unknown-version error message tested and pasted into the PR body
• unavailable versions skipped and recorded, not silently dropped
• experiments/gate-v1/README.md and docs/gate/runner.md open questions updated
• Zero-token dry-run rows still labelled as not-a-measurement
• Branch track1/b4-matrix-pins