Context
The gate task currently recorded is: open login page, fill username, fill password, submit, click Dashboards. Four steps, all on the login surface.
That is too thin to measure, for two independent reasons:
- PRD section 8 picked a long task deliberately: "the flow is long enough (8–12 steps) to make step-level validity statistics meaningful." With four steps, one broken locator moves step-validity by 25 percentage points. The number would be noise.
docs/gate/testbed.md Open questions raises the sharper objection: "Whether B2 gate task (login + navigate) stays browser-meaningful once Grafana HTTP APIs cover the same clicks — Track-1 must pick tasks that still stress DOM locators, not only API-equivalent config." Login-and-navigate is the least DOM-interesting thing Grafana does.
This is an expensive-to-reverse choice — everything downstream (recorder, compiler, replay, the whole matrix run, the gate memo) is built on it, and re-recording across eight versions is not cheap. CONTRIBUTING says expensive-to-reverse choices become ADRs.
What to produce
docs/decisions/ADR-0006-track1-gate-task.md, following the shape of ADR-0003 (Status / Context / Options considered with honest case for and against / Decision / Consequences / Reversal cost / Open questions).
The decision must specify:
- The task, step by step, in 8–12 steps, expressed as intents rather than selectors (selectors belong to the recorder). Candidate surfaces worth evaluating — pick on evidence, not on this ordering:
- Create a dashboard, add a panel, choose a visualisation, configure a query against the seeded TestData datasource, set a panel title, save, and verify it appears in the dashboards list.
- Create an alert rule with a notification contact point and verify it appears in the rules list — closest to the PRD section 8 "first ugly task", and heavy on multi-step forms.
- Add a datasource through the settings panel including the Save-and-test round trip, which exercises async validation.
- Why each step is DOM-meaningful. For every step, state what would make it hard to do via the HTTP API, or explicitly concede it is API-equivalent and justify why it stays. A task where all twelve steps are trivially API-equivalent makes the churn measurement vacuous, which is exactly the ADR-0003 open question.
- Assertability. Per PRD section 9 and the pivot brief's selection rule test 5, every step needs a crisp observable end state. Name the post-condition for each step and whether you expect the compiler to synthesize it
strong or weak (see the strength table in docs/gate/compiler.md). A task that is mostly weak assertions cannot support the gate.
- Cross-version viability. The task must exist in a recognisable form on every available matrix version. Grafana 9.5's dashboard editor and Grafana 13's are different UIs — that is the churn we want, but if a step has no counterpart at all on one end, the task is unrunnable there and the matrix shrinks. Check this against the versions the tag-verification issue confirmed, and say which versions you actually opened.
- Parameterisation. Which values become typed parameters (
contracts/trajectory.schema.json parameters) — dashboard name, panel title, query expression. Recall that the recorder never stores literal values, only slots.
Constraints
- Do not write recorder code in this PR. This is the decision artifact; the recording is a separate issue that cites it.
- Do the work by hand first: bring up a testbed instance, click the candidate task on the oldest and newest available versions, and write what you saw. An ADR reasoned from documentation instead of from the running UI will be wrong in the way that costs the most later.
- No invented metrics. "Roughly N steps" is fine; "expected 85% survival" is a fireable error per CONTRIBUTING.
How to test
npm run testbed -- --version <oldest available>
# perform the candidate task by hand; note every step, every observable end state
npm run testbed -- --version <oldest available> --down
npm run testbed -- --version <newest available>
# repeat; note what moved, renamed, or disappeared
npm run testbed -- --version <newest available> --down
npm run lint:docs
Before you open the PR
Context
The gate task currently recorded is: open login page, fill username, fill password, submit, click Dashboards. Four steps, all on the login surface.
That is too thin to measure, for two independent reasons:
docs/gate/testbed.mdOpen questions raises the sharper objection: "Whether B2 gate task (login + navigate) stays browser-meaningful once Grafana HTTP APIs cover the same clicks — Track-1 must pick tasks that still stress DOM locators, not only API-equivalent config." Login-and-navigate is the least DOM-interesting thing Grafana does.This is an expensive-to-reverse choice — everything downstream (recorder, compiler, replay, the whole matrix run, the gate memo) is built on it, and re-recording across eight versions is not cheap. CONTRIBUTING says expensive-to-reverse choices become ADRs.
What to produce
docs/decisions/ADR-0006-track1-gate-task.md, following the shape of ADR-0003 (Status / Context / Options considered with honest case for and against / Decision / Consequences / Reversal cost / Open questions).The decision must specify:
strongorweak(see the strength table indocs/gate/compiler.md). A task that is mostlyweakassertions cannot support the gate.contracts/trajectory.schema.jsonparameters) — dashboard name, panel title, query expression. Recall that the recorder never stores literal values, only slots.Constraints
How to test
Before you open the PR
docs/README.mdADR table row addedtrack1/b2-gate-task-adr