Skip to content

v0.12.0 — a result is served only while the inputs it ran on are still yours

Choose a tag to compare

@manuglopez manuglopez released this 01 Oct 09:41
· 7 commits to main since this release
a07eac5

A result is served only while the inputs it ran on are still the inputs you have

Until now, a result layer trusted that every result in it ran on its commit's clean tree, and nothing checked it. Three ways that broke, each reproduced as a run reporting green on a suite plain PHPUnit fails:

  • Uncommitted edits, then a branch switch. A run on a dirty tree stored its results under the clean commit. Discard the edit, git checkout -b other, and the next run replayed them.
  • Results written without the commit moving, then a revert. A CI pass without --allow-ci-baseline, or an interrupted pass, wrote results into the layer but left its commit behind. A revert to that commit gave an empty diff, and the stale results were served. This also happened when only a watched data file had changed.
  • A dirty layer, published. With remote_push: all, a layer recorded on uncommitted edits reached the shared remote. A clean machine adopted it and replayed it.

What changed

  • Every result is stamped with its inputs: the content key it ran under, plus a digest of the inputs no edge captures (watched files, migrations, Blade templates, sibling and fallback scopes). Before anything is replayed, both are checked against the current tree. A result whose stamp no longer matches executes, and --explain says why: StaleResult content key changed, non-edge inputs changed, or unstamped.
  • No other test can move your stamp. A test's scopes exclude its own dependencies, so another test gaining or losing an edge leaves its digest alone.
  • The graph is published only from a clean tree. Objects are always publishable, because they are addressed by the content that actually ran. A shared object now holds up to 20 digest variants, so machines with different non-edge inputs no longer crowd each other out.

Two selection holes, closed

  • A .php file excluded from coverage (<source><exclude>: typically providers, middleware, kernels and the exception handler) has no edges, so editing it selected no test. It now runs every test, whatever watch pattern also names it. To narrow that, take the file out of <source><exclude> so coverage records who executes it.
  • A watch pattern never fired for a file that already had edges. The edge rule claimed the file first, and the pattern was silently ignored. Watch patterns now only ever add. They apply on top of edges, they never narrow the run-everything rule above, and a key spelled like one of the Laravel fallbacks (resources/views/**, database/migrations/**) adds to that fallback instead of replacing it.

Also fixed:

  • A migration for a table no test records now selects every test that uses the database.
  • @includeFirst is recognised.
  • A root file with a numeric name (2024) no longer crashes every command.

Costs, stated plainly

  • One-time: results recorded before 0.12 carry no stamp, so they execute once and are then stamped. On a large suite that is one full pass.
  • Per pass: on a real Laravel application (726 test files, 2,194 source files), checking stamps adds about 0.35 s to a warm pass and 0.75 s to the first pass on a machine. A persistent, racy-safe content-hash cache keeps it there.
  • Selection is a little broader where watch patterns now apply on top of edges: a config/, routes/ or lang/ edit runs every test its pattern maps to, about 3–6% more on that application.

Content keys, fingerprint composition and remote paths are unchanged.