Skip to content

ditto v0.11.0

Choose a tag to compare

@Disble Disble released this 19 Sep 17:51
· 9 commits to main since this release

Gated() answers the complete module scope

Gated() compiled the mutated file's own package with go test -c, which is a narrower question than the default go test -count=1 ./.... A mutant in package P that only a test in a package importing P can kill survived it — and nothing in the output could show the difference, because the survivor addresses matched and the run was faster either way.

A gated run now discovers the module's package layout once, compiles the complete default scope's test binaries, and runs each of them for every selected mutant.

Gating replaces only the commands whose complete execution plan has been measured: go test -count=1 ./... and its built-in -json form in either flag order. Any custom or unsupported WithTestCommand — a package-local scope, an extra flag, extra spacing, an alternate executable — keeps the ordinary laboratory and the runner exactly as configured, counted as fallen back. The set is closed on purpose: a command that is almost the default is not the default.

A module whose test binaries collide now gates

The module-scope runner named every test binary after its package, so on a tree where two packages produced the same name — a root module, cmd/x and internal/x all build x.test — it refused before compiling anything, gating fell back for every file, and the repository gated nothing while looking like any other gated run. Measured on ditto itself: 0 of 850 mutants compiled.

Packages are now planned into batches free of name collisions — case-folded on Windows, where the filesystem is — and each batch compiles into its own directory. A collision-free module still plans exactly one batch, so the one-compilation win is unchanged.

Verification

Checked on the tagged commit 0d315b211b3e7ea56f704e44e1eee78057714075:

  • CI, both matrix legs (Go 1.27 and latest): pass
  • CodeQL, OSV-Scanner, SonarCloud: pass
  • chained mutation gate over the change: 85 mutants, 74 killed, 11 survived, score 0.87 against a 0.50 minimum
  • local gate: lint 0 issues, 522 tests passed, 15 exact counters passed

The first pull request was red on the Linux runner with validateBatches - goos always receives "linux" while a Windows host reported nothing: the parameter is the seam that makes Windows naming testable on a Linux host, and its only varying call sites were tests passing the literal "linux". Reproduced locally with GOOS=linux golangci-lint run, fixed by covering the Windows case-fold rather than by suppressing the finding, and re-verified at 0 issues under that target. Recorded in docs/learning-log.md.

What this release does not claim

The module path was measured, not assumed, and the follow-up work that did not earn its place is recorded as refutation:

  • shared compilation plus fail-fast semantics removed 208 package starts from a real ten-mutant run and was still slower in every rotated round (1.1284, 1.0801, 1.0429)
  • outer parallelism with an explicit two-worker scheduler overlapped exactly and preserved every verdict, but returned only 5.62% (ratio 0.943784 against a pre-registered 0.75 limit)
  • the configured test command is 99.3% of a real staged run, so no precision-preserving headroom was measured inside ditto for that scope

Known open items, stated rather than implied as fixed: a module-path deadline kill is reported as an assertion; sources that are not gofmt-clean get no gating and are reported as none; and the heavy-suite ratio is unmeasured at this revision.

The Gated: none of 85 line in this repository's own CI gate is the admission rule working: that gate runs a configured command, not the default, so every mutant keeps its own path.