v1.6.2 — Xcode projects with package plugins, and no score that was not earned

[ericodx]

This release makes Xcode projects whose packages apply a build tool plugin testable, and stops the tool from printing a score it did not earn. Before, xcodebuild stopped at Validate plug-in waiting for a trust prompt a headless build cannot answer: every mutant came back unviable and the run printed 100%. Now plugin and macro validation is skipped, a project that does not build stops the run with the compiler error, a run where no mutant could be tested shows n/a, and a build that runs out of time stops the run instead of counting its mutants as detected. A new --xcodebuild-arg flag passes extra arguments to every xcodebuild call. Thanks to @mifkata for the report and for testing every fix on a real project (#176).

---

Bug fixes

Xcode projects with package plugins (#176)

• Every build-for-testing passes -skipPackagePluginValidation and -skipMacroValidation. A project whose packages apply a build tool plugin, such as SwiftLint or the OpenAPI generator, no longer stalls at Validate plug-in
• Every source the tool writes into the sandbox is wrapped in // swiftlint:disable all and // swiftlint:enable all. A SwiftLint build tool plugin used to lint the schematized code and fail the build on its generated identifiers, which made every mutant unviable. The integrity check compares the files without the wrapper

No score that was not earned (#176)

• When the schematized build produces nothing, the project is built once without mutations. If that build fails, the run exits 1 with the compiler output, which --keep-logs also writes to baseline.log. Before, every mutant was reported unviable
• A run where no mutant could be tested has no mutation score. It shows n/a in the console, HTML and Markdown reports, and --min-score and --max-score-drop fail on it. Before, it printed 100%
• A fallback build, or a warm or per-mutant build of an incompatible mutant, that does not finish within --build-timeout stops the run with Build did not finish within Ns. Raise --build-timeout. It used to record its mutants as Timeout, which counts as detected, so a project whose fallback builds all ran out of time reported 100%

Timed-out Xcode builds (#176)

• A timed-out build no longer leaves its sandbox behind. SWBBuildService and the swift-frontend processes it starts run in their own process groups, so the signal sent to xcodebuild's group missed them, and they kept writing into the sandbox after the tool had removed it (225 MB to 1.3 GB per run). They are now stopped and killed first

Cache

• The cache records the extra xcodebuild arguments, so changing them tests every mutant again
• The cache format is now version 4. Caches written by older versions can hold false unviable verdicts from the plugin validation, which would never be tested again. They are discarded once, with the usual warning

---

New

Extra xcodebuild arguments (#176)

• --xcodebuild-arg <arg>, repeatable, and the xcodebuild-args key in .swift-mutation-testing.yml pass arguments to every xcodebuild call of a run: the shared build, the test runs, the fallback builds, and the builds and tests of incompatible mutants
• Arguments the tool sets itself, such as the actions, -scheme, -destination or -derivedDataPath, are refused with exit code 1. So is the flag on a package, since SwiftPM builds take no xcodebuild argument
• The command line replaces the file's list as a whole, and an argument that takes a value is two items: the flag, then its value

swift-mutation-testing --scheme App --destination "platform=macOS" \
  --xcodebuild-arg -allowProvisioningUpdates \
  --xcodebuild-arg OTHER_SWIFT_FLAGS='$(inherited) -DMUTATION_TESTING'

xcodebuild-args:
  - -allowProvisioningUpdates
  - -xcconfig
  - Config/Mutation.xcconfig

---

Upgrading

• The first run after upgrading tests every mutant again. Cache format 4 discards the older caches once
• A project that does not build now ends the run with 1. Before, it finished with every mutant unviable and a 100% score
• A build timeout now ends the run with 1. If a fallback or incompatible-mutant build needs more time than the default 120 s, raise --build-timeout
• The score can be n/a. A quality gate with --min-score or --max-score-drop fails when no mutant could be tested, and a baseline written from such a run has no score

---

Test coverage

• 1509 tests in 181 suites
• 9583 of 9586 source lines covered
• 3775 of 3783 regions covered (99.79%)

This repository's own score on the default tier is still 99.8%, from the self-run of 2026-10-13: 1672 mutants, 98.8% over every operator. That is the README badge.

---

Known issues

• The Xcode path does not validate a baseline; a failing suite there still makes every mutant look killed
• Signed-Releases and reproducible build provenance are not in place
• A mutant that hangs pays the targeted timeout and then the full one

---

Requirements

• macOS 15+
• Swift 6.2+ (Xcode 26)
• Xcode project or workspace with a valid scheme and test target, or an SPM package with a test target

---

Installation

See the Installation Guide for Homebrew, pre-built binary, and build from source instructions.

---

What's Changed

• Update the pre-commit hook versions by @ericodx in Update the pre-commit hook versions #173
• Move the campaign and benchmark scripts under .github/scripts by @ericodx in Move the campaign and benchmark scripts under .github/scripts #174
• chore: move SECURITY.md to the repository root by @ericodx in chore: move SECURITY.md to the repository root #175
• Skip package plugin validation, pass extra xcodebuild arguments and stop when the unmutated build fails by @ericodx in Skip package plugin validation, pass extra xcodebuild arguments and stop when the unmutated build fails #177

Full Changelog: v1.6.1...v1.6.2

---

This discussion was created from the release v1.6.2 — Xcode projects with package plugins, and no score that was not earned.