Make the v3 rehearsal pass from a clean checkout - #106
Conversation
The clean-checkout rehearsal required by PUBLISH_READINESS.md before any sota-v3 spend could not pass. `_stage_site_inputs` symlinks `ROOT/web/node_modules` only when it already exists, which is true in a working tree and never true in a fresh clone. `bun run build` then exited 127 because vite was absent, and `check=True` aborted the whole rehearsal on an unhandled CalledProcessError with the stderr swallowed by capture_output. The gate had been passing for the wrong reason: the 2026-07-27 run inherited node_modules from the tree it ran in, and the one dependency it never checked was the one a clean-checkout rerun exists to rule out. No test caught it because every case in tests/test_sota_v3_rehearsal.py passes run_web_build=False, leaving the build path with no coverage. `_ensure_web_dependencies` now installs with --frozen-lockfile when node_modules is missing, reuses it when present, and raises with the installer's stderr. The committed bun.lock stays authoritative so the fetch resolves pinned versions and cannot drift the built site; "zero spend" still means no provider or model call. What happened is recorded under web_build.dependencies in the report. Adds regression coverage for the install, reuse, and failure paths. Touches no file in _CONTRACT_SOURCES; fingerprint remains a523bdfcebe47bbd.
The rehearsal now passes unassisted from a `--no-local` clone with no web/node_modules at 63f28e6: status passed, zero spend, seven mutations rejected, sota_v2 rejected / sota_v3 accepted, site data byte-matching the checked-in frozen v2 dataset, dependencies installed from the committed lockfile, staged build green, 738 tests passing in the same clone. Checks the pre-spend clean-checkout box and keeps a short account of the first failed attempt, since "the gate passed because it inherited state from the tree it ran in" is the failure this program exists to catch. Docs only. Fingerprint remains a523bdfcebe47bbd. No spend authorized; every lane gate stays false.
|
Warning Review limit reached
Next review available in: 59 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
This PR fixes the SOTA-v3 “clean checkout” rehearsal so it can successfully run the staged web/ build without inheriting web/node_modules from an already-prepared working tree, and records a verified run in the publish-readiness checklist.
Changes:
- Add
_ensure_web_dependenciesto installweb/node_moduleswithbun install --frozen-lockfilewhen missing and record the outcome underweb_build.dependencies. - Update
_run_web_buildto include dependency-resolution details in the rehearsal report. - Add regression tests for install/reuse/failure dependency paths, and document the verified clean-checkout run in
docs/PUBLISH_READINESS.md.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| scripts/sota_v3_rehearsal.py | Ensures staged web/ dependencies exist in clean clones and records dependency state in the web build report. |
| tests/test_sota_v3_rehearsal.py | Adds tests covering install, reuse, and failure behavior for staged web dependencies. |
| docs/PUBLISH_READINESS.md | Marks the clean-checkout rehearsal as verified and documents the failure mode + fix. |
Suppressed comments (1)
scripts/sota_v3_rehearsal.py:537
subprocess.run(..., check=True, capture_output=True)will raiseCalledProcessErroron build failures, but the traceback message typically omits the captured stdout/stderr, which makes failures non-actionable (similar to the original node_modules issue). Consider usingcheck=Falseand raising aRuntimeErrorthat includes a stderr/stdout tail on non-zero exit.
check=True,
capture_output=True,
text=True,
)
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if completed.returncode != 0: | ||
| raise RuntimeError( | ||
| f"`{bun} install --frozen-lockfile` failed in {staging / 'web'} " | ||
| f"(exit {completed.returncode}); the staged site cannot be built.\n" | ||
| f"{completed.stderr.strip()}" | ||
| ) |
Three pre-spend readiness gaps, none touching a _CONTRACT_SOURCES file: The scaffold-view checkbox in PUBLISH_READINESS.md was marked complete while its own body cited fingerprint 4f6ddddd6a6dd81c, and a later paragraph in the same document required a rerun under the current fingerprint. Rerunning is free and deterministic, so the compare was re-run under a523bdfcebe47bbd. Every headline mean, all eight per-seed scores, and the paired t reproduce exactly; neither scripted agent reads a field the intervening contract changes touched. The gate is now closed for this contract and the contradiction is resolved rather than papered over. The rehearsal's clean-checkout fix landed in #106 with three tests that all call _ensure_web_dependencies directly, so deleting the call site inside _run_web_build left the suite green while silently restoring the exit-127 regression on a fresh clone. Added two tests that drive _run_web_build itself and assert the install precedes the build. Verified by mutation: stubbing out the wiring line fails 2 of 11 tests, where it previously failed 0 of 9.
What
The clean-checkout rehearsal that
docs/PUBLISH_READINESS.mdrequires before anysota-v3spend could not pass. This fixes it and records the verified run.The defect
_stage_site_inputssymlinksROOT/web/node_modulesinto the staging copy only when it already exists. That is true in a working tree and never true in a fresh clone.bun run buildthen exited 127 becausevitewas absent, andcheck=Trueaborted the whole rehearsal on an unhandledCalledProcessErrorwith the stderr swallowed bycapture_output— a traceback that said "exit status 127" and nothing aboutnode_modules.So the 2026-07-27 rehearsal passed by inheriting
node_modulesfrom the tree it ran in. The gate was passing for the wrong reason, and the one dependency it silently inherited was exactly the condition a clean-checkout rerun exists to rule out. No test caught it: every case intests/test_sota_v3_rehearsal.pypassesrun_web_build=False, so the build path had no coverage at all.The fix
_ensure_web_dependenciesinstalls with--frozen-lockfilewhennode_modulesis absent, reuses it when present, and raises with the installer's stderr instead of a bareCalledProcessError. What happened is recorded underweb_build.dependenciesin the report.The committed
bun.lockstays authoritative, so the fetch resolves pinned versions and cannot drift the built site. "Zero spend" continues to mean no provider or model call; a package fetch touches no contract source.Three regression tests cover the install, reuse, and failure paths.
Verification
Unassisted run from a
--no-localclone with noweb/node_modulesat63f28e6:status: passed,spend_usd: 0.0,evidence_class: synthetic-non-evidencewrong-contract,soft-fallback,stale-scaffold,unknown-version-dispatch,unregistered-route,tampered-compact-score,raw-link-mismatch)sota_v2rejected /sota_v3accepteddependencies: installed(40 packages), staged build greenScope
Touches no file in
_CONTRACT_SOURCES. Fingerprint remainsa523bdfcebe47bbd, matchingconfig/sota_v3_lane.json. Authorizes no spend; every lane gate stays false.Part of #93.