test: a stale build cannot answer for the tree - #74
Merged
Conversation
vite preview SERVES web/build/. It does not compile, playwright starts it, so
pnpm exec playwright test after an edit to web/src drives the previous build and
reports on code that no longer exists. just test-e2e never had the problem
because it depends on build; the trap is the shortcut everybody reaches for to
run one spec, and the config's own 'run just build first' comment is only read
by somebody who already suspects.
Measured on one tree, three ways, by renaming the reader's 'next brick' control,
which two cases in reader.test.ts assert on by name:
no rebuild, no guard 18 passed
no rebuild, guard refuses
rebuild 2 failed
The first line is the point. The suite did not go quiet, it answered
confidently about a tree nobody has.
web/playwright.setup.ts is a globalSetup that compares the newest build output
against everything the build is made from and throws with the offending paths
and the command to run. server/crates is on that list: a Rust change reaches the
browser through just wasm, and it is the staleness nobody suspects, because
nothing under web/ was touched. Cargo.toml and Cargo.lock too, since a
dependency bump changes the wasm without changing a line of Rust.
mortar-wasm/pkg/ is excluded, and finding out why was the substance of this.
It lives inside web/src but just wasm rewrites every file in it on every
invocation whether or not any Rust changed, and just check depends on wasm. Left
in, the local gate would leave the tree looking stale to every playwright run
after it, forever. Its real source is server/crates, which is watched, so a Rust
change still fires and a rebuild that changed nothing does not. Verified in that
order: build, then just wasm, then a Rust touch.
Nine vitest cases against temporary directories, because this guard fails OPEN:
if a path moves and the walk finds nothing it reports a fresh build forever and
hands back exactly the behaviour it exists to prevent. One of them pins the pkg
exclusion from both sides, asserting the false positive is real without it.
Contributor
|
🧹 Preview wall removed. |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
vite previewSERVESweb/build/. It does not compile, playwright starts it foryou, so
pnpm exec playwright testafter an edit toweb/srcdrives theprevious build and reports on code that no longer exists.
just test-e2enever had this problem, because it depends onbuild. The trapis the shortcut everybody reaches for to run one spec, and the config's own
"run
just buildfirst" comment is only ever read by somebody who alreadysuspects.
Measured, on one tree, three ways
Rename the reader's "next brick" control, which two cases in
reader.test.tsassert on by name:
The first line is the whole point. The suite did not go quiet or flake. It
answered confidently, in ten seconds, about a tree nobody has.
(My first attempt at this reproduction renamed "previous brick", which no case
asserts on, so its green run was correct and proved nothing. Worth saying,
because a demonstration that cannot fail is the same class of mistake as the bug
it is demonstrating.)
The guard
web/playwright.setup.tsis aglobalSetupthat compares the newest thing inweb/build/against everything the build is made from, and throws with theoffending paths and the command to run.
server/cratesis on the input list: a Rust change reaches the browser throughjust wasm, and that is the staleness nobody suspects, because nothing underweb/was touched.Cargo.tomlandCargo.locktoo, since a dependency bumpchanges the wasm without changing a line of Rust.
It stays quiet during the loop that matters:
web/tests/is not a build input,so editing a spec and rerunning it, over and over, never trips it.
The part that took the work
mortar-wasm/pkg/is excluded, and finding out why is most of what this PRis.
It lives inside
web/src, so it looks like an input. Butjust wasmrewritesevery file in it on every invocation whether or not any Rust changed, and
just checkdepends onwasm. Left in, the local gate would leave the treelooking stale to every playwright run after it, forever, for a reason nobody
could act on. Measured:
pkg/mtime...778beforejust wasm,...830after,against a build at
...782.The exclusion is safe rather than convenient because
pkg/is output. Its realsource is
server/crates, which is watched, so a Rust change still fires theguard and a rebuild that changed nothing does not. Verified in that order:
just build(silent),just wasm(still silent),touch feed.rs(refuses,naming
feed.rs).Verification
Nine vitest cases against temporary directories, because this guard fails
open: if a path moves and the walk quietly finds nothing, it reports a fresh
build forever and hands back exactly the behaviour it exists to prevent. A guard
nothing exercises is the one that stops guarding silently.
One case pins the
pkg/exclusion from both sides, asserting that the falsepositive is real without it, so the exclusion cannot be dropped as decoration.
just checkgreen (159 Rust, 215 vitest, both tsc projects, four guards) and 74Playwright cases.
Documented where people meet it:
AGENTS.mdand.specs/development-guidelines.md.🤖 Generated with Claude Code