tooling: check.sh runs no browser suite, and the real-server cycle run was three times the only detector

[JArmandoAnaya]

The gap

bash scripts/check.sh — the canonical "before you say it works" invocation, named as such in CONTRIBUTING.md and CLAUDE.md — runs no Playwright suite at all. It covers pytest, vitest, ruff, mypy, import-linter, eslint, and the four drift gates (openapi, generated client, wire fixtures, mcp tools). It does not start a browser.

There are three suites. Two of them are invisible to check.sh:

Suite	Command (from frontend/app)	CI job
unit / drift	bash scripts/check.sh (repo root)	python, frontend
app + annotator e2e	CI=1 npx playwright test	annotator e2e (chromium)
full cycle, real server	CI=1 npx playwright test -c playwright.cycle.config.ts	browser cycle (chromium)

Why this matters more than it sounds

During the 2026-08 audit remediation run (PRs #304–#313), the real-server cycle suite was three separate times the only suite to catch a regression:

• The annotator opens as a viewer when nothing can be written (F2, F23) #306 (T3) — Finish job stayed disabled over a job that was finished. The job's allowed_actions was never invalidated after a save, so the button kept the answer it got when every asset was still unannotated. check.sh was green; the annotator e2e never drives a job to completion.
• Make promotion observable (F5, F17, F18) #308 (T5) — the promote button's only feedback was its own label flipping to "Promoted". The cycle spec asserted that flip, so removing it (the actual fix for finding F5) failed there and nowhere else.
• Review flow UI: the half of the progress machine that had no door (F24) #309 (T6) — the annotator's job counter ran backwards when a frame was accepted: the readout was literally counts.annotated, and accepting moves an asset out of that bucket. Unreachable until the review flow existed; the cycle run found it on the first pass, 3 of 3 becoming 2 of 3.

In each case a reasonable person would have reported the work as verified on a green check.sh. One of them (#304) actually shipped that way and went red in CI.

Proposal

1. check.sh runs all three suites by default, or — if the ~4 minutes of browser time is judged too expensive for the common case — offers them via a flag and prints a loud, unmissable line when it skips them. Silence is the problem: "All checks passed" currently means "all the checks this script knows about", and nothing says which those are.
2. Document the CI=1 requirement in the script header and in CONTRIBUTING.md. playwright.config.ts sets reuseExistingServer: !process.env.CI, so a stale vite server on port 5273 answers instead of the build under test. During the run this produced six failures in unrelated demo scenarios that read as genuine code bugs and cost about fifteen minutes to diagnose.
3. Consider a check.sh browser subset alongside the existing python|frontend|generated subsets, so the browser suites are addressable by name.

Notes

• annotator bench (chromium, manual) is workflow_dispatch-only and must stay out of any default — it is deliberately not a required check.
• The rule itself has been added to the refactor-protocol skill's testing section, so an agent or a person reading it before a task now runs all three. This issue is about making the script honest rather than relying on the reader remembering.

[JArmandoAnaya]

Related: #319 — the cycle suite is currently flaky (an auto-start race in the annotator surfaces INVALID_TRANSITION where the spec waits for "Saved"; seen on main after #311 and on the docs-only #318).

That does not weaken this proposal, it sharpens the order: fix #319 first, then make check.sh run the suite. Wiring a flaky required suite into the canonical command teaches people to re-run it without reading it, which is worse than not running it at all.