Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions .claude/hooks/testing-policy.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
#!/usr/bin/env bash
# SessionStart hook — state the verification-cost policy for cloud sessions.
#
# A cloud session starts with no memory of what the last one learned the hard
# way, and the most expensive lesson in this repository is about the browser
# gate: `npm run verify:ui` is ~25 minutes, CI repeats it wholesale on every
# change that touches a browser surface, and a session that does not know this
# will spend those 25 minutes buying a verdict GitHub is about to reach anyway.
# Measured 2026-09-02 on two consecutive changes: 25.0m and 20.5m locally, while
# the focused selection for the same diffs took 37s and 6.1s and agreed.
#
# So this hook says, once per session and in the model's own context, which gate
# to reach for first. It is scoped to cloud sessions because that is where the
# ask came from and where the session has no local history to draw on; a
# workstation session already has the docs, the receipts store, and the operator.
#
# Contract: READ-ONLY and unfailable. Emits nothing but `additionalContext` on
# stdout, exits 0 on every path including a malformed payload, and makes no
# decision — a session that ignores it simply runs the wider gate, which is the
# conservative outcome. SessionStart hook stdout is injected into context;
# stderr is not, which is why every line below goes to stdout.
set -uo pipefail

# Cloud sessions only. `session-start.sh` uses the same gate.
[ "${CLAUDE_CODE_REMOTE:-}" = "true" ] || exit 0

# Drain stdin so the caller never blocks on an unread pipe.
cat >/dev/null 2>&1 || true

root="${CLAUDE_PROJECT_DIR:-}"
[ -z "$root" ] && root="$(git rev-parse --show-toplevel 2>/dev/null || true)"
[ -z "$root" ] && root="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." 2>/dev/null && pwd || true)"
# Without the planner there is no advice to give, and inventing some would be
# worse than silence.
[ -f "$root/scripts/browser-test-plan.mjs" ] || exit 0

read -r -d '' context <<'POLICY' || true
[testing-policy] Verification cost policy for this cloud session.

CI re-runs the browser suite on every change that touches a browser surface, so a
full local `npm run verify:ui` (~25 min, 646 tests) buys a verdict GitHub is about
to reach. Do not spend it by default.

Before any browser run, ask the planner which part of the suite the diff can break:

npm run plan:browser # dry run: the level, the specs, and why
npm run plan:browser -- --run # execute the plan it printed

It fails closed: shared foundations, an unattributable UI file, or unknown scope
all escalate to the full suite on their own. You do not have to judge that.

For the static gates the arbiter already answers the same question:

npm run arbiter -- <gate> # RUN / DEFER / PROVEN, with its evidence

Reporting rules, which the cost saving depends on:
- A narrowed run is NOT the full gate. Say "focused browser proof at level <x>,
full suite left to CI" — never "verify:ui passed".
- A deferred gate is not a passed gate; say "deferred to CI".
- Paste the decisive line of real output. Exit 0 alone is not proof.

Unchanged by any of this: GitHub remains the authoritative merge gate and runs
exactly what it ran before. Never weaken a required check to save local time.
POLICY

# The context is a JSON string field, so the only characters that must be escaped
# are backslash and double quote; the heredoc above contains neither, and this
# substitution keeps that true if it ever does. Newlines become \n.
escaped="${context//\\/\\\\}"
escaped="${escaped//\"/\\\"}"
escaped="${escaped//$'\n'/\\n}"

printf '{"hookSpecificOutput":{"hookEventName":"SessionStart","additionalContext":"%s"}}\n' "$escaped"
exit 0
12 changes: 12 additions & 0 deletions .claude/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,13 @@
"Bash(npm run verify:pr-local)",
"Bash(npm run verify:pr-local --*)",
"Bash(npm run verify:phone-chrome)",
"Bash(npm run plan:browser)",
"Bash(npm run plan:browser --*)",
"Bash(npm run arbiter)",
"Bash(npm run arbiter --*)",
"Bash(npm run arbiter:status)",
"Bash(npm run receipts)",
"Bash(npm run check:browser-test-plan)",
"Bash(npm run ensure)",
"Bash(npm run skills)",
"Bash(npm run docs:check-index)",
Expand Down Expand Up @@ -305,6 +312,11 @@
"type": "command",
"command": "bash \"$CLAUDE_PROJECT_DIR/.claude/hooks/issues-surface.sh\"",
"timeout": 30
},
{
"type": "command",
"command": "bash \"$CLAUDE_PROJECT_DIR/.claude/hooks/testing-policy.sh\"",
"timeout": 15
}
]
}
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ For the verification principle, the tier table, and the rest of the gate-selecti

## Do not pay twice for the verdict GitHub is about to reach

For the rule against re-deriving a verdict GitHub is about to reach, and the gate arbiter's inputs and non-negotiable boundaries, see [`docs/agents/verification-gates.md`](docs/agents/verification-gates.md).
For the rule against re-deriving a verdict GitHub is about to reach, the gate arbiter's inputs and non-negotiable boundaries, and the browser-gate planner that narrows `verify:ui` to the specs a diff can actually break (`npm run plan:browser`), see [`docs/agents/verification-gates.md`](docs/agents/verification-gates.md).
<!-- END:process-hardening -->

<!-- BEGIN:page-and-button-wiring -->
Expand Down
69 changes: 69 additions & 0 deletions docs/agents/verification-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,72 @@ twice. The smallest-correct-gate rule above still decides which gate is right; t
only decides whether that gate has anything left to tell you before you push.

<!-- END:verification-gates -->

## The browser gate is narrowed, not deferred

`npm run verify:ui` is the most expensive run here — 646 Chromium tests, ~25 minutes —
and CI repeats it wholesale: `Production UI critical` and the three `Production UI`
shards are guarded on `ui_changed`, so any change touching a browser surface gets the
full suite on GitHub whether or not it ran locally first.

The arbiter above cannot help with it. Its lever is deferral, and `ui` is in
`NEVER_DEFER_CLASSES` — pushing a UI change with **no** browser evidence is not a bet
this repository takes, and that stays true. The lever here is a different one: run the
part of the suite the diff can actually break.

```bash
npm run plan:browser # dry run: the level it chose, the specs, and why
npm run plan:browser -- --run # execute that plan
npm run plan:browser -- --full # force the whole suite
```

Four levels, and only the middle two are a saving:

| Level | When | What runs |
| --------- | -------------------------------------------------------- | -------------------------------------------------------------------------------------- |
| `none` | no browser surface changed | nothing — CI skips Production UI for this scope too |
| `changed` | a `ui-*.spec.ts` changed | those specs, **complete** — never grepped, since the diff rewrote their own assertions |
| `focused` | changed UI source attributable to specs | the owning specs, complete |
| `full` | shared foundation, unattributable file, or unknown scope | `verify:ui` |

**Attribution is evidence, not a table.** A spec owns a changed file when both contain
the same literal — a `data-testid` the source renders and the spec asks for, or a route
the source defines and the spec navigates to. Component names are deliberately not
matched: a spec never names a component, so a match there would be a comment, and a
comment is not proof a journey covers the code.

**It fails closed, the opposite of the arbiter.** A file with no owning spec, a shared
foundation (`globals.css`, any shared style root, the Playwright config, the runner, the
shell/chrome coordinator set), a browser-lane path it cannot classify, or a deleted
file all escalate to the full suite on their own. The arbiter's bug costs a redundant
run; this one's would cost an unrun journey, so the defaults point the other way.

**The project is part of the selection.** `chromium` grep-inverts `@mockup` and
`chromium-mockups` collects only those, so a mockup spec run under `--project=chromium`
collects nothing and "passes" having executed no test. The planner reads both
`testMatch` patterns out of `playwright.config.ts` (rather than copying them, which
would drift) and routes each selected spec accordingly — `tests/ui-tools.spec.ts` holds
both kinds and gets both projects. A spec neither project collects escalates to the full
suite rather than producing a command that matches nothing.

**Coverage that rests on an unreadable precondition is reported as conditional.**
`ui-critical-fast` is guarded on `github.event.pull_request.draft != true`, which no
worktree can evaluate. On a draft PR CI skips the very job that makes narrowing safe, so
the planner prints those assumptions with the verdict instead of stating flatly that CI
will repeat the run.

Non-negotiable, and the reason the saving is allowed at all:

- **A narrowed run is not the UI gate.** Report it as "focused browser proof at level
`<x>`, full suite left to CI" — never "verify:ui passed". `check:browser-test-plan`
and `tests/browser-test-plan.test.ts` pin that wording in the runner itself.
- **CI is untouched.** The planner advises local work only; GitHub runs exactly what it
ran before, and no required check may be weakened to save local time.
- **Dry run by default.** `--run` executes; nothing happens without it.
- **When CI will NOT repeat it** — a browser-lane change on a scope where the Production
UI jobs are skipped — the planner says so and tells you to consider `--full`, because
there the local run is the only browser evidence there will be.

Cloud sessions are told this at SessionStart by `.claude/hooks/testing-policy.sh`, which
is read-only, exits 0 on every path, and states the reporting rules alongside the
commands.
2 changes: 1 addition & 1 deletion docs/scripts-index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Scripts index

Curated map of `scripts/` (283 files) and the `package.json` script surface (284 entries),
Curated map of `scripts/` (284 files) and the `package.json` script surface (286 entries),
grouped by purpose. This is orientation, not an exhaustive per-file listing — the authoritative
command list is `package.json`, and `npm run docs:check-scripts` verifies every `npm run <x>`
referenced in docs resolves to a real script. `npm run docs:update` refreshes the exact counts above.
Expand Down
4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
"test:coverage": "node scripts/run-vitest.mjs run --coverage",
"test:coverage:node": "node scripts/run-vitest.mjs run --project=node --coverage",
"test:coverage:ui": "node scripts/run-vitest.mjs run --project=jsdom --coverage",
"test:ci-workflows": "node scripts/run-vitest.mjs run tests/ci-cache-safety.test.ts tests/authenticated-live-workflow.test.ts tests/codex-autofix-workflow.test.ts tests/codex-run-pr-operator-workflow.test.ts tests/eval-canary-workflow.test.ts tests/live-drift-workflow.test.ts tests/ops-digest.test.ts tests/container-ci-contract.test.ts tests/test-runner-safety.test.ts tests/installed-lock-parity.test.ts tests/railway-config.test.ts tests/ingestion-autopilot.test.ts tests/ingestion-autopilot-workflow.test.ts tests/check-lighthouse-budget.test.ts tests/live-web-vitals-inputs.test.ts tests/offline-release-profile.test.ts",
"test:ci-workflows": "node scripts/run-vitest.mjs run tests/ci-cache-safety.test.ts tests/authenticated-live-workflow.test.ts tests/browser-test-plan.test.ts tests/codex-autofix-workflow.test.ts tests/codex-run-pr-operator-workflow.test.ts tests/eval-canary-workflow.test.ts tests/live-drift-workflow.test.ts tests/ops-digest.test.ts tests/container-ci-contract.test.ts tests/test-runner-safety.test.ts tests/installed-lock-parity.test.ts tests/railway-config.test.ts tests/ingestion-autopilot.test.ts tests/ingestion-autopilot-workflow.test.ts tests/check-lighthouse-budget.test.ts tests/live-web-vitals-inputs.test.ts tests/offline-release-profile.test.ts",
"test:cc-guards": "node scripts/run-vitest.mjs run --reporter=dot tests/caring-contacts-plan-draft.dom.test.tsx tests/caring-contacts-plan-patient-detail.test.ts tests/caring-contacts-plan-activation.test.ts tests/caring-contacts-plan-wizard.dom.test.tsx tests/caring-contacts-schedule.test.ts tests/caring-contacts-schedule-view.test.ts tests/caring-contacts-schedule-route.test.ts tests/caring-contacts-schedule-screen.dom.test.tsx tests/caring-contacts-schedule-page.dom.test.tsx tests/caring-contacts-clock.test.ts tests/caring-contacts-new-plan-page.dom.test.tsx tests/caring-contacts-explained-automation.dom.test.tsx tests/caring-contacts-workspace-shell.dom.test.tsx tests/caring-contacts-patients-directory.dom.test.tsx tests/caring-contacts-patient-overview.dom.test.tsx tests/caring-contacts-patients-page.dom.test.tsx tests/caring-contacts-domain-isolation.test.ts tests/caring-contacts-interface-vocabulary.test.ts tests/caring-contacts-retention.test.ts tests/caring-contacts-repository.test.ts tests/caring-contacts-overlay-definitions.test.ts tests/caring-contacts-overlay-trigger-inventory.test.ts tests/caring-contacts-workspace-screens.test.ts tests/route-reachability.test.ts tests/design-system-adoption.test.ts tests/caring-contacts-contact-time-adjustment.dom.test.tsx tests/caring-contacts-contact-route.test.ts tests/caring-contacts-overlay-trigger.dom.test.tsx tests/caring-contacts-overlay-host.dom.test.tsx tests/source-control-bytes.test.ts tests/caring-contacts-demo-seed.test.ts tests/caring-contacts-pathway-versions.test.ts tests/caring-contacts-templates-library.dom.test.tsx tests/caring-contacts-templates-page.dom.test.tsx tests/caring-contacts-template-detail.dom.test.tsx tests/caring-contacts-template-detail-page.dom.test.tsx tests/caring-contacts-reporting.test.ts tests/caring-contacts-guidance-reports-pages.dom.test.tsx tests/caring-contacts-team-workload.test.ts tests/caring-contacts-team-route.test.ts tests/caring-contacts-team-roster.dom.test.tsx tests/caring-contacts-team-page.dom.test.tsx",
"test:e2e": "node scripts/run-playwright.mjs",
"test:e2e:all": "node scripts/run-playwright.mjs",
Expand Down Expand Up @@ -86,6 +86,8 @@
"verify:cheap:internal": "npm run check:runtime && npm run check:installed-lock-parity && npm run check:upload-limit-parity && npm run check:github-actions && npm run check:ci-scope && npm run check:verification-plan && npm run check:gitleaks-pinned && npm run check:ci-triage && npm run check:pr-policy && npm run check:gate-manifest && npm run check:skills && npm run check:branch-review-ledger && npm run check:outstanding-issues && npm run check:ledger-write-discipline && npm run check:pr-mergeability && npm run sitemap:check && npm run check:repo-awareness-snapshot && npm run docs:check-index && npm run docs:check-inventory && npm run docs:check-scripts && npm run docs:check-links && npm run check:knip && npm run check:maintainability-budgets && npm run brand:check && npm run check:assets && npm run check:therapy-data-index && npm run check:cross-mode-index && npm run check:mha-act-sections && npm run check:type-scale && npm run check:icon-scale && npm run check:design-system-contract && npm run check:migration-role && npm run check:function-grants && npm run check:owner-scope && npm run lint && npm run typecheck && npm run test",
"verify:pr-local": "node scripts/verify-pr-local.mjs",
"verify:phone-chrome": "node scripts/verify-phone-chrome.mjs",
"plan:browser": "node scripts/browser-test-plan.mjs",
"check:browser-test-plan": "node scripts/browser-test-plan.mjs --self-test",
"audit:final-merge": "node scripts/final-merge-audit.mjs",
"audit:merge-loss": "node scripts/audit-merge-loss.mjs --self-test && node scripts/audit-merge-loss.mjs",
"verify:ui": "npm run check:runtime && npm run check:installed-lock-parity && npm run test:e2e:pr",
Expand Down
Loading
Loading