v0.13.0 — migrations select by whether your tests actually run them

Migrations select by whether your tests actually run them

Until 0.12, every test that used a database trait was credited with every table named in any migration. Any migration change ran the whole database suite, whichever table it touched. The table tracker also started after setUp(), so it never saw tables touched by trait setUps, factories in traits or seeding. That blind spot is why the blanket union was needed.

0.13 records what each test really queries, and asks what a change does to the test database:

• A migration already squashed into the test connection's schema dump never runs in tests, so editing it selects nothing. --explain says so.
• A pending migration runs in every fresh test database, so it selects every database test.
• A schema-dump change selects every database test, unless only comments, whitespace or migration ids changed. A change to the set of migration names counts: squashing a data migration silently removes its rows from test databases.

Also new

• schema_dump => 'per-table' (opt-in) narrows a dump change to the tests of the changed tables. It follows foreign keys in both directions, and triggers, routines and views.
• Migrations in every directory the app loads them from: the migration_paths config, literal loadMigrationsFrom() calls, and stancl/tenancy's database/migrations/tenant.
• A new file in a new subdirectory of a sibling directory (app/Console/Commands/Reports/…) selects the tests next to it.
• Table tracking starts before any provider boots, any trait setUp or any seeding. It also works under the in-process extension.
• The extractor understands unions, comma joins, multi-table TRUNCATE/UPDATE, DELETE … USING and aliases.
• migrations => 'conservative' restores the old behaviour.

Costs, stated plainly

• One full re-record on upgrade. Structural fingerprint keys change, and the digest moves from n3 to n4.
• Recording overhead is about +1.5 to 2.5 s per million queries.
• Known limits: tables read only through static caches, raw PDO, queries in a provider's register(), and stored functions called without FROM.

v0.12.4 — works with PHPUnit 13.4

Works with PHPUnit 13.4

PHPUnit 13.4.0 changed the constructors of three configuration classes that phpunit-replay builds itself: XmlConfiguration\Loader, CliArguments\Builder and Configuration\Merger. Each now takes the event emitter. On 13.4 every command printed:

phpunit-replay: unexpected error (Too few arguments to function PHPUnit\TextUI\XmlConfiguration\Loader::__construct() …): degrading to a plain PHPUnit run

The suite still ran in full, so nothing was reported green that should not have been. But nothing was recorded or replayed either, which made every run a full run.

phpunit-replay now passes the emitter when the constructor asks for one, and nothing otherwise. PHPUnit 11.5, 12 and 13.0–13.4 are all supported.

If you updated to PHPUnit 13.4 and saw the "degrading to a plain PHPUnit run" line, upgrade to 0.12.4.

Known: parallel runs on PHPUnit 13.4 need a newer ParaTest

ParaTest 7.25.0 (the latest at the time of writing) installs alongside PHPUnit 13.4 but cannot drive it: it builds PhpHandler without the argument 13.4 now requires. This affects ParaTest with or without phpunit-replay. Until ParaTest ships support, a --parallel run on PHPUnit 13.4 now ends with one clear line instead of a bare fatal error:

phpunit-replay: ParaTest v7.25.0 cannot run on PHPUnit 13.4.0 (an upstream incompatibility, not a test failure). Run without --parallel, or pin phpunit/phpunit below 13.4.0 until ParaTest supports it.

Nothing is recorded or replayed from such a pass. Sequential runs work on 13.4.

v0.12.3 — record accepts --allow-ci-baseline, and a misplaced flag fails clearly

record accepts --allow-ci-baseline, and a misplaced flag fails clearly

Seeding a baseline from CI with record failed:

vendor/bin/phpunit-replay record --fresh -p --allow-ci-baseline
phpunit-replay: unexpected error (Unknown option "--allow-ci-baseline". Most similar options are --ignore-baseline, --generate-baseline, --use-baseline, ...

Only run defined --allow-ci-baseline. The flag was forwarded to PHPUnit, whose error pointed at its own, unrelated baseline options.

What changed

• record --allow-ci-baseline works like it does on run: in CI, the recording finalizes and publishes the branch baseline.

• verify deliberately does not take the flag. It never finalizes or publishes a baseline.

• A phpunit-replay flag on the wrong command now stops before PHPUnit, with a message that names the commands it belongs to:

  phpunit-replay: --allow-ci-baseline is an option of "run" and "record", not of "verify"
  

  Options that take a value (they may be PHPUnit's own, such as --log-junit) are still forwarded. So is everything after --.

v0.12.2 — a structural change no longer announces a recording that may not happen

A structural change no longer announces a recording that may not happen

When phpunit-replay.php, composer.lock or phpunit.xml changed, run printed:

phpunit-replay: structural change (replay_config): the cached baseline cannot be used, recording a fresh baseline
Replay  ✓ 0 executed (0 affected, 0 uncached) · 11029 replayed (11029 from remote) · …

The first line was false. The warning was printed as soon as the local graph was found stale, and before the run looked at the shared remote. When CI had already recorded a baseline under the new configuration, the run adopted it and recorded nothing.

What changed

Messages only. The same baselines are adopted or recorded in exactly the same cases as before.

• The structural-change line now only states the fact: structural change (X): the cached baseline cannot be used.
• When a baseline is adopted from the remote, the run says so: adopted the <branch> baseline (<sha>) from the remote.
• recording a fresh baseline is printed only when nothing usable was found: no remote, a miss, or a remote baseline that is stale too.
• --dry-run prints the same lines as the real run.

v0.12.1 — one erroring test no longer turns every run into a full recording

One erroring test no longer turns every run into a full recording

PHPUnit exits 2 as soon as one test errors, for example with an uncaught exception or a missing table, and 1 for a failure. phpunit-replay counted only 0 and 1 as a pass that ran to the end. A record with a single erroring test therefore saved its results without a baseline. The next pass found no cached baseline and recorded the whole suite again, and so did every pass after it, for as long as that test kept erroring.

On a real application, three erroring tests in one file were enough to turn a no-change run into a full recording of 10,957 tests. record gave no hint of it, because it still printed baseline <branch>@<sha> for a baseline it had not saved.

What changed

• An error is an outcome, like a failure. A pass that exits 0, 1 or 2 now counts as complete. The erroring test is recorded, and it re-runs on the next pass like any failure. Only a run that stopped early (aborted or truncated) or never finished (a crash or a signal) is incomplete. The in-process path never had this problem.
• record only names a baseline it saved. When a pass finalizes no baseline, the summary line leaves out the baseline segment instead of showing HEAD. That happens with a CI run without --allow-ci-baseline, or with an aborted pass.

Nothing is invalidated. Content keys, stamps, fingerprints and the remote layout are unchanged.

v0.12.0 — a result is served only while the inputs it ran on are still yours

A result is served only while the inputs it ran on are still the inputs you have

Until now, a result layer trusted that every result in it ran on its commit's clean tree, and nothing checked it. Three ways that broke, each reproduced as a run reporting green on a suite plain PHPUnit fails:

• Uncommitted edits, then a branch switch. A run on a dirty tree stored its results under the clean commit. Discard the edit, git checkout -b other, and the next run replayed them.
• Results written without the commit moving, then a revert. A CI pass without --allow-ci-baseline, or an interrupted pass, wrote results into the layer but left its commit behind. A revert to that commit gave an empty diff, and the stale results were served. This also happened when only a watched data file had changed.
• A dirty layer, published. With remote_push: all, a layer recorded on uncommitted edits reached the shared remote. A clean machine adopted it and replayed it.

What changed

• Every result is stamped with its inputs: the content key it ran under, plus a digest of the inputs no edge captures (watched files, migrations, Blade templates, sibling and fallback scopes). Before anything is replayed, both are checked against the current tree. A result whose stamp no longer matches executes, and --explain says why: StaleResult content key changed, non-edge inputs changed, or unstamped.
• No other test can move your stamp. A test's scopes exclude its own dependencies, so another test gaining or losing an edge leaves its digest alone.
• The graph is published only from a clean tree. Objects are always publishable, because they are addressed by the content that actually ran. A shared object now holds up to 20 digest variants, so machines with different non-edge inputs no longer crowd each other out.

Two selection holes, closed

• A .php file excluded from coverage (<source><exclude>: typically providers, middleware, kernels and the exception handler) has no edges, so editing it selected no test. It now runs every test, whatever watch pattern also names it. To narrow that, take the file out of <source><exclude> so coverage records who executes it.
• A watch pattern never fired for a file that already had edges. The edge rule claimed the file first, and the pattern was silently ignored. Watch patterns now only ever add. They apply on top of edges, they never narrow the run-everything rule above, and a key spelled like one of the Laravel fallbacks (resources/views/**, database/migrations/**) adds to that fallback instead of replacing it.

Also fixed:

• A migration for a table no test records now selects every test that uses the database.
• @includeFirst is recognised.
• A root file with a numeric name (2024) no longer crashes every command.

Costs, stated plainly

• One-time: results recorded before 0.12 carry no stamp, so they execute once and are then stamped. On a large suite that is one full pass.
• Per pass: on a real Laravel application (726 test files, 2,194 source files), checking stamps adds about 0.35 s to a warm pass and 0.75 s to the first pass on a machine. A persistent, racy-safe content-hash cache keeps it there.
• Selection is a little broader where watch patterns now apply on top of edges: a config/, routes/ or lang/ edit runs every test its pattern maps to, about 3–6% more on that application.

Content keys, fingerprint composition and remote paths are unchanged.

v0.11.0 — a branch no longer replays a pass its own diff never vouched for

A branch no longer replays a pass its own diff never vouched for

The graph keeps one result layer per branch, each recorded at a commit. A run takes its diff against one baseline, the nearest one, but it used to serve results from other layers too. The branch's own layer was laid over the nearest baseline's, and the fallback chain added more, without checking that the served result's layer matched the tree the diff was taken from.

Here is the false green that exposed it. A feature branch runs MoneyTest, and the passes land in its own layer. It then reverts that change and merges a develop that broke Money::add. develop wins as the nearest baseline, the diff is empty, and the branch's stale passes override develop's failures:

Replay  ✓ 0 executed … 35 replayed

That run exits 0 while PHPUnit fails five tests. The same class appeared when the nearest baseline's commit came from the shared remote.

What changed

• Every layer is audited before anything is served. The layer whose commit is the diff base is trusted as that commit's tree. Any other layer gets its own diff. Whatever the rules would select from that diff is not served from that layer. A layer with no commit, or with one git does not have, serves nothing: that costs a cache miss, never a stale hit. A test left with no valid result executes, and --explain says why:

  tests/MoneyTest.php  ← StaleLayer  src/Money.php (feature@abc1234)
  

• Publishing is tied to the key. push, and the pushes that follow a run (from the CLI and in-process), publish a result only when its stored content key equals the key recomputed now. Before, leftovers from an incomplete pass could be published as a verdict on content they never ran against.

• The last-run snapshot narrows a diff only when it describes that layer and that commit.

• verify measures divergences against what run would actually serve.

Content keys, fingerprints, the remote layout and the graph format are unchanged. The one-time cost is that a branch whose own layer no longer matches re-runs those tests once, then re-bases.

Known, and not closed here

Two related cases reproduce on 0.10.x and are still open:

• results written into a layer without its commit moving (a CI pass without --allow-ci-baseline, or an incomplete pass), followed by a revert;
• results recorded on a working tree with uncommitted changes, then read from another branch or published with remote_push: all.

They share one root: a layer does not know which tree each result ran on. They will be fixed together, by stamping every result with the inputs it ran against.

v0.10.2 — an environment change no longer runs nothing and calls it green

After an environment change, a run ran nothing and said it passed

When the environment changes (PHP minor, coverage driver, OS or coverage format), phpunit-replay throws away the cached results on purpose: a result recorded on PHP 8.3 says nothing about PHP 8.4. It keeps the graph and its edges, because which files a test depends on does not change with the environment.

The run list is built from what changed and from what holds no result. It had no bucket for a test file the graph knows that simply has no result any more. Nothing selected those files, and there was nothing to replay, so the run printed

phpunit-replay: environment change (driver): cached results cleared
Replay  ✓ 0 executed (0 affected, 0 uncached) · 0 replayed · 0 quarantined

and exited 0 on a suite that plain PHPUnit fails. It never recovered on its own either: the stored fingerprint was never updated, so every later run found the same drift and did the same thing, until --fresh. A routine PHP minor upgrade was enough to trigger it.

What changed

• A known test file never goes both unexecuted and unreplayed. Any test file the graph knows that holds no result in the layers the pass serves from now executes. --explain shows it as Uncached (no cached result). The rule is not specific to environment drift: it covers any way results go missing, such as an interrupted record or a cleared layer. It applies to both the CLI pipeline and the in-process path.
• It recovers on its own. Clearing results on drift now updates the stored fingerprint to the current environment. The first pass after a change executes and records, the next one replays, and the warning no longer repeats forever.

Content keys, fingerprint composition, the remote layout and the graph format are unchanged. No cache is invalidated beyond the one the environment change already cleared.

v0.10.1 — a shared cache no longer vouches for changes its key never saw

A shared cache could replay a test that a change had broken

A test file is served from a remote object when an object exists under its content key k. That key is built from the fingerprint, the test file's hash and the hashes of the files the test executed.

Some tests are selected for other reasons: a watch pattern (including the residue fallback), or the Laravel Migration, Blade or Sibling rules. For those, the thing that triggered the selection is none of the inputs above. A watched data file the test reads is not a file it executed, so changing it left k as it was. The object still matched, and the run printed

✓ 0 executed (0 affected, 0 uncached) · 36 replayed (5 from remote)

on a suite that plain PHPUnit fails. Without a remote, the same change executes and fails correctly. The bug only affected teams sharing a cache.

What changed

• A test file is served from the remote only when every reason it was selected is one that k covers: PhpEdge or TestFile. The allowed reasons are an explicit list, so a rule added in future makes the test execute until someone decides k covers it. The check applies both to the CLI pipeline and to the in-process path.
• --explain and --dry-run used to list only what remained after the remote had served its share, so the rule looked as if it had never fired. They now show the whole selection with each file's reason, and mark the files a remote object stands in for with [served from remote].

Content keys, fingerprints and the remote layout are unchanged. No cache is invalidated, and objects already published keep serving the selections they always could.

v0.10.0 — remote:init, and a rejected push no longer eats its deletions

Sharing the cache stops being a document you read

Every remote shape worked, and every one of them needed a manual setup a user had to find, read and translate. That is where adoption dies — not in the number of options, but in the absence of a command.

$ vendor/bin/phpunit-replay remote:init

origin:      git@github.com:acme/widget.git
cache:       git@github.com:acme/widget-replay-cache.git
private:     always — the cache stores test names, file paths and failure messages

create:      created acme/widget-replay-cache with `gh repo create --private`

probe:       published objects/2026-09/replay-probe-f7cbc5e8ab5f8bad.json
             read it back from a separate clone — the round trip works
             probe deleted

config:      wrote phpunit-replay.php

The probe is the point. "Configured" and "works" are different claims, and only the second is worth printing. The command builds the real cache backend through the factory, publishes an object, reads it back through a separate clone, compares, and deletes it.

What it does by not doing it

Never touches a credential	gh already owns yours. A test-tooling package that asks for a personal access token is a package nobody installs. Without gh it prints the exact manual steps and continues rather than failing
Never writes a secret into your repository	CI publishing needs a deploy key in the cache repo plus a secret in your own — two writes to real infrastructure. It prints the steps and a workflow snippet and stops
Never creates a public repository	and there is no --public
Never edits an existing config	an existing phpunit-replay.php is left byte-identical and the lines to add are printed. A config with baseline_branches already tuned must not be clobbered by a setup command

--dry-run prints every action and takes none. --same-repo configures the other supported shape — an orphan branch of the repository you already have, nothing to create and no access to grant — and prints the cost that keeps it from being the default: a plain git clone of your project fetches every branch, so the cache lands in everyone's clone.

Known gap, pinned rather than closed: --name value as two tokens is not recognised, only --name=value. The argv pre-splitter recognises a value-taking option only with its value attached by =, the same as run --log-junit and prune --keep-months already; closing it changes shared behaviour and belongs in its own change.

A rejected push no longer eats its deletions

prune --remote silently discarded every deletion it made whenever the push was rejected once, and then reported success — printing a count of objects it had not removed.

doEnd() commits the deletion; a concurrent client rejects the push; adoptFetchedHistory() does git reset --hard FETCH_HEAD, which discards the local commit and restores the deleted file exactly as upstream still holds it. The helper then replayed the write buffer, which knew nothing about deletions, so nothing was staged and the retried push succeeded with no error.

Root cause in one sentence: the reconciliation helper could replay a run's writes but had no representation of its deletions.

Fixed with a tombstone set beside the write buffer. The cancellations in both directions are the part a naive fix gets wrong — delete() also drops any buffered put() of that key, put() also drops any tombstone for it. A delete() of a key absent from the local mirror tombstones too, deliberately: the mirror is only as fresh as its last refresh, so a key can be absent here and present upstream, and gating on local presence would reintroduce the same bug through staleness instead of push timing.

Nothing was ever corrupted — this is garbage collection and a later run retried — but it was silent, which is what the retry loop exists to prevent. Four of the six new tests fail against unfixed code.

It was found because remote:init needed to delete its own probe object and could not. A command written to prove the round trip works proved something else first.

Upgrading

Nothing to do. No configuration key was added, renamed or given a new default, and no cache is invalidated.

Full changelog: v0.9.0...v0.10.0