Skip to content

v1.2.0 — Granular cache invalidation and up to 87% faster re-execution

Choose a tag to compare

@ericodx ericodx released this 08 Apr 12:35
3271825

Re-execution times drop by up to 80% when only a subset of test files change. The cache now tracks which test file killed each mutant and invalidates only the affected entries when tests are added, modified, or removed.


What's new

Granular cache invalidation

  • Per-file test hashing replaces the previous all-or-nothing project hash — only mutants whose killer test file changed are re-executed
  • TestFileDiff tracks three change categories: added, modified, and removed test files between runs
  • KillerTestFileResolver maps killer test names back to their source file paths, supporting both XCTest class names and Swift Testing function names
  • CacheStore.invalidate(diff:) applies status-aware rules: .killed and .survived entries are invalidated when their associated test file changes; .unviable and .killedByCrash are permanent and never invalidated
  • New test files (additions) invalidate all .survived entries — a new test might kill a previously surviving mutant
  • Removed test files invalidate .killed entries whose killer was in the deleted file
  • Modified test files invalidate both .killed and .survived entries linked to that file

Cache metadata persistence

  • CacheStore now stores killerTestFile alongside each cached result
  • TestFilesHasher.hashPerFile produces a [String: String] dictionary of relative paths to SHA256 content hashes
  • Test file hashes are persisted in the cache directory and compared on subsequent runs to compute TestFileDiff
  • Symlinked test files outside the project root use absolute paths as keys to avoid collisions

Invalidation pipeline in MutantExecutor

  • prepareCacheStore loads the cache, computes current test file hashes, detects changes via changedTestFiles(current:), and triggers granular invalidation before execution begins
  • The pipeline runs automatically — no configuration needed, no CLI flags to set

Architecture changes

Cache layer — expanded from simple key-value storage to a metadata-aware invalidation system:

Type Responsibility
TestFileDiff Value type representing added, modified, and removed test files between runs
TestFilesHasher Computes per-file SHA256 hashes for all test files under the project
KillerTestFileResolver Resolves test names (XCTest or Swift Testing) to their source file paths
CacheStore Extended with invalidate(diff:), changedTestFiles(current:), and killerTestFile metadata

Execution layerExecutionDeps now includes killerTestFileResolver. TestExecutionStage, IncompatibleMutantExecutor, and FallbackExecutor all resolve and store killerTestFile with each result.

Cache key formatMutantCacheKey no longer includes a global test files hash. The key is now derived solely from the mutant's source file content and operator, making it stable across test-only changes.


Performance

Measured on the project's own codebase (1145 mutants, SPM mode):

Scenario Duration vs. first run
First run (cold cache) ~98 min
Re-run, no changes ~13 min −87%
Re-run, 1–2 test files changed ~17 min −83%
Re-run, all test files changed ~98 min same as cold

The 87% reduction on cache-hit runs comes from skipping build and test execution entirely for cached mutants — only mutants with invalidated cache entries are re-executed.


Bug fixes

  • Fixed simulator slot released after result parsing instead of before — prevents slot starvation under high concurrency
  • Fixed duplicated result recording logic in TestExecutionStage — extracted shared recordResult method

Internal improvements

  • 36 commits with granular test coverage for every new type and invalidation rule
  • Shared test fixtures extracted across all test suites (MutantDescriptor, ExecutionResult, MutantCacheKey, RunnerConfiguration, BuildArtifact, and stage-specific fixtures)
  • Test fixture deduplication across cache, CLI, sandbox, discovery, execution, and reporting test suites
  • TestFilesHasher covers edge cases: external symlinks, non-existent paths, non-test file exclusion

Requirements

  • macOS 15+
  • Swift 6.2+
  • Xcode project with a valid scheme and test target, or an SPM package with a test target

Installation

See the Installation Guide for Homebrew, pre-built binary, and build from source instructions.


What's Changed

  • feat: granular test cache invalidation by @ericodx in #56
  • docs: remove mutation score badge by @ericodx in #57
  • refactor: reduce duplication and increase test coverage by @ericodx in #58

Full Changelog: v1.1.0...v1.2.0