v0.7.0 — what ditto's answers are worth

A release about ditto's own answers: whether they can be believed, and whether the gate that produces them ever finishes. Both were measured rather than assumed, and both were wrong.

Breaking

The mutation score changes value for the same code. A mutant that never compiled used to be counted as killed — a failing compile exits non-zero, and that is how ditto recognises a kill — and it now leaves the numerator and the denominator.

Measured on a fixture with two non-viable mutants of five: 0.60 → 0.33. One earned kill of one viable mutant, which is what the tests actually did. If a WithMinimumThreshold was tuned against an older score, expect it to move — toward the truth.

On this repository, 83 of 748 mutants do not build: 11.1%, against a benchmark of Major 1.8% and PIT 0%.

The default test command gains -json. It is what lets ditto say why a mutant died.

The headline

A verdict now carries its reason. Ditto read any non-zero exit as a killed mutant — a failed assertion, a mutant that never compiled, a timeout, a missing binary. Measured on one file, 78 mutants: 22% of the kills were not assertions.

And ditto had no deadline anywhere. A test binary invoked directly takes -test.timeout 0 — disabled — so on the gated path a mutant that loops never returned. Both paths are bounded now, and a mutant the deadline stops is a kill carrying its own reason, as PIT, Stryker and Infection all do.

make test.mutation.staged is a gate that measures the change and finishes: 4 mutants, 82 seconds, green, against 424 of 727 that die at thirty minutes. Not a smaller timeout — a smaller question.

Which checks ran on the tagged commit, and which did not

Green by push event on bbef658: Continuous Integration (both legs, 1.27 and latest), CodeQL, OSV-Scanner, SonarCloud.

Red: the Mutation Tests workflow. It times out at 1800s having reached 424 of 727 mutants, and it has failed on every push since 2026-08-27 — 0.4.0, 0.5.0 and 0.6.0 all shipped over it. That is stated here rather than left to be discovered.

It is not a regression and it is not a mystery. It is measured three times, with both levers spent: gating removes 54% of the compilations and does not close it, and cutting the mutant's suite by 46% moved the gate by 0.5%, because -failfast already stops a killed mutant at its first failing test. The bill is the wrong size rather than badly paid, and docs/backlog.md entry 21 carries it.

Full notes in CHANGELOG.md, and the measurements behind each claim in docs/experiments/.