Skip to content

v0.11.0 — a branch no longer replays a pass its own diff never vouched for

Choose a tag to compare

@manuglopez manuglopez released this 30 Sep 18:23
· 8 commits to main since this release
8c087c7

A branch no longer replays a pass its own diff never vouched for

The graph keeps one result layer per branch, each recorded at a commit. A run takes its diff against one baseline, the nearest one, but it used to serve results from other layers too. The branch's own layer was laid over the nearest baseline's, and the fallback chain added more, without checking that the served result's layer matched the tree the diff was taken from.

Here is the false green that exposed it. A feature branch runs MoneyTest, and the passes land in its own layer. It then reverts that change and merges a develop that broke Money::add. develop wins as the nearest baseline, the diff is empty, and the branch's stale passes override develop's failures:

Replay  ✓ 0 executed … 35 replayed

That run exits 0 while PHPUnit fails five tests. The same class appeared when the nearest baseline's commit came from the shared remote.

What changed

  • Every layer is audited before anything is served. The layer whose commit is the diff base is trusted as that commit's tree. Any other layer gets its own diff. Whatever the rules would select from that diff is not served from that layer. A layer with no commit, or with one git does not have, serves nothing: that costs a cache miss, never a stale hit. A test left with no valid result executes, and --explain says why:

    tests/MoneyTest.php  ← StaleLayer  src/Money.php (feature@abc1234)
    
  • Publishing is tied to the key. push, and the pushes that follow a run (from the CLI and in-process), publish a result only when its stored content key equals the key recomputed now. Before, leftovers from an incomplete pass could be published as a verdict on content they never ran against.

  • The last-run snapshot narrows a diff only when it describes that layer and that commit.

  • verify measures divergences against what run would actually serve.

Content keys, fingerprints, the remote layout and the graph format are unchanged. The one-time cost is that a branch whose own layer no longer matches re-runs those tests once, then re-bases.

Known, and not closed here

Two related cases reproduce on 0.10.x and are still open:

  • results written into a layer without its commit moving (a CI pass without --allow-ci-baseline, or an incomplete pass), followed by a revert;
  • results recorded on a working tree with uncommitted changes, then read from another branch or published with remote_push: all.

They share one root: a layer does not know which tree each result ran on. They will be fixed together, by stamping every result with the inputs it ran against.