Skip to content

nwyin/irradiate

Repository files navigation

irradiate

Fast mutation testing for Python, written in Rust.

Why

Mutation testing is slow. The bottleneck isn't generating mutants — it's running the test suite once per mutant. A typical pytest startup costs 200-500ms, and with hundreds of mutants that adds up to minutes of pure overhead.

irradiate eliminates this by maintaining a pool of pre-warmed pytest workers. Pytest starts once, collects tests once, then forks a child process for each mutant. The result: mutation testing at 30-60 mutants/sec on real codebases.

How it works

  1. Parse Python source with tree-sitter (27 mutation operator categories, ~160+ distinct mutations)
  2. Generate trampolined mutants — each function gets an original, N mutated variants, and a runtime dispatcher
  3. Collect test coverage and timing in a single pytest run
  4. Fork a child process per mutant inside pre-warmed workers (no pytest restart)
  5. Report results as terminal output, JSON (Stryker schema v2), HTML, or GitHub Actions annotations

Install

pip install irradiate

Or build from source:

cargo build --release

Requires Python 3.10+ with pytest installed.

Usage

# Run mutation testing (auto-detects src/ and tests/)
irradiate run

# Only test functions changed since main
irradiate run --diff main

# Generate JSON report (Stryker mutation-testing-report-schema v2)
irradiate run --report json

# Generate self-contained HTML report
irradiate run --report html

# Fail CI if mutation score is below threshold
irradiate run --fail-under 80

# See cached results
irradiate results

# Show diff for a specific mutant
irradiate show module.x_func__irradiate_1

Configuration

Configure via [tool.irradiate] in pyproject.toml:

[tool.irradiate]
paths_to_mutate = "src"
tests_dir = "tests"
do_not_mutate = ["**/generated/*", "**/vendor/*"]
pytest_add_cli_args = ["-x", "--tb=short"]

All settings can be overridden via CLI flags. Run irradiate run --help for the full list.

Features

Mutation operators (27 categories)

Arithmetic, comparison, boolean, augmented assignment, unary, string mutation/emptying, number literals, lambda bodies, return values, assignments, default arguments, argument removal, method swaps, dict kwargs, decorator removal (planned), exception types, match/case removal, condition negation, condition replacement, statement deletion, keyword swap, loop mutation, ternary swap, slice index removal.

Functions can be excluded with # pragma: no mutate.

Execution model

  • Fork-per-mutant (default): Workers fork after pytest collection. Each mutant runs in an isolated child process — no state leakage between mutants, no pytest restart overhead.
  • --isolate: Full subprocess isolation. Slower but guaranteed clean for projects with complex test infrastructure.
  • --verify-survivors: After the main run, re-tests survived mutants in isolate mode to catch false negatives from warm-session state leakage.

Incremental mode (--diff)

Only mutate functions touched by a git diff. Uses git merge-base to compare against the divergence point, so --diff main does the right thing on feature branches.

Reporting

Caching

Content-addressable cache keyed on SHA-256 of function body + test IDs + operator. Survives rebases, branch switches, and touch — unlike mtime-based caches.

Decorator support

@property, @classmethod, and @staticmethod are handled natively via a descriptor-aware trampoline. Other decorated functions are skipped (source-patching fallback planned — see #13).

Performance tuning

  • --workers N: control parallelism (defaults to CPU count)
  • --timeout-multiplier N: scale per-mutant timeout (default 10x baseline)
  • --worker-recycle-after N: respawn workers after N mutants (auto-tuned)
  • --max-worker-memory N: recycle workers exceeding N MB RSS
  • --covered-only: skip mutants with no test coverage
  • --no-stats: skip coverage collection, test all mutants against all tests

How it compares to mutmut

mutmut irradiate
Speed pytest.main() per mutant (~200ms each) Fork-per-mutant — pytest starts once
Parser LibCST (Python) tree-sitter (Rust, parallel)
Operators ~20 categories 27 categories
Cache mtime-based Content-addressable (SHA-256)
Orchestration Python multiprocessing Rust + tokio async
Incremental --diff with merge-base
Reports Terminal only JSON, HTML, GitHub Actions annotations
Decorator support Skip all @property/@classmethod/@staticmethod handled
CI integration Manual --fail-under, GitHub annotations, step summary
Isolation Fork only Warm-session + --isolate + --verify-survivors

Acknowledgments

irradiate's trampoline architecture and mutation operator design are informed by mutmut. The naming convention is partially compatible with mutmut to ease migration.

License

MIT

About

Fast mutation testing for Python, written in Rust

Topics

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors