pytest-leela 0.9.0

Fixed

• The clean baseline reads current local source, tests and helpers.
  Timestamp bytecode caches could reuse stale code after a same-size edit
  with an unchanged mtime, accepting a failing baseline and running mutants.
  Leela now removes only the project's source and assertion-rewrite caches
  before coverage and the unmutated baseline, leaving installed packages alone.

• Installed packages in a virtualenv inside the project are no longer
  evicted between mutants. Leela evicted every module whose __file__
  sat under the cwd (and, before a run, everything under the temp dir).
  With uv's default .venv inside the project that included third-party
  packages, so C extensions such as numpy failed to re-import with
  ImportError: cannot load module more than once per process. That
  crash was scored as a kill, so every mutant reported killed,
  including mutants no test caught. Eviction now skips anything under
  sys.prefix, sys.base_prefix, sys.exec_prefix,
  site.getsitepackages(), site.getusersitepackages() or a
  site-packages / dist-packages directory, while still evicting the
  project's own modules so mutated code reloads.

  Scores reported by earlier versions may be inflated for projects
  whose virtualenv lives inside the project directory. Re-run to get an
  honest number.

• Database tests under pytest-django can run inside leela's inner
  sessions. pytest-django blocks DB access at configure time and only
  restores it at unconfigure, which is after leela runs. Each inner
  session's DjangoDbBlocker recorded that outer blocking wrapper as the
  "real" ensure_connection, so django_db_blocker.unblock() could
  never reach the database: every DB test errored in setup, and each
  setup error was scored as a kill, including for mutants no test
  could detect. Leela now lifts the outer block (via pytest-django's own
  unblock()) while it runs. Scores from earlier versions on
  pytest-django projects with DB tests may be inflated.

• Django model and admin modules are no longer reloaded between
  mutants. Re-executing them re-registered models ("Reloading models is
  not advised") and admin classes (AlreadyRegistered), and re-entered
  import cycles that only resolve in Django's app-loading order. Any
  project module they reference is kept too, so a kept model never ends
  up subclassing a stale copy of a reloaded mixin.

• The summary no longer reports a negative "pruned by type analysis"
  count. A typed rule can add operators the untyped rule lacks (int
  + gains //), and count_pruned subtracted list lengths, so such
  points counted as negative pruning ("-1 pruned"). It now counts only
  untyped mutations the typed rule dropped.

• Under --diff, the summary's candidate, pruned and type-enrichment
  counts cover only the changed lines. They counted every mutation
  point in the target files while only changed-line mutants ran ("159
  candidates" for 88 tested). The diff filter now applies before type
  enrichment, counting and generation.

Added

• Clean baseline before any mutant. The selected tests run once with
  no mutation applied, through the same in-process path as every mutant.
  If anything fails or errors, the run aborts with a message naming the
  failing tests and a non-zero exit, instead of scoring every mutant as
  killed.

• A third mutant status, error. A mutant whose inner run crashed,
  exited abnormally (usage error, nothing collected, interrupted) with no
  test failure or collection failure, or ran zero tests, was previously
  reported as killed (crash) or SURVIVED (tests_run: 0). It is now
  an error: reported with its reason in the terminal, HTML and JSON
  reports, excluded from the mutation score, and failing the session by
  default. MutantResult gains error and a status of "killed",
  "survived" or "error"; RunResult gains errors and
  mutants_scored. Timeouts remain kills.

• A test module that fails to collect under the mutant is a kill.
  The clean baseline shows it collects unmutated, so the failure is the
  mutant's doing: the target raising at import, or a test module's own
  module-level check failing. Those modules are the killing tests.
  MutatingLoader records exceptions raised by the mutated source, and a
  conftest that fails to import because of one is a kill too. A
  conftest's own failing check, or a conftest or test module that calls
  pytest.exit() at import, is an error: it is not attributed to the
  mutant. So is a conftest collection hook that calls pytest.exit();
  the error names the node being collected. Error text renders each
  collection failure as path: Type: message. A run that
  stays green or empty is not a kill: an import caught by the test is
  SURVIVED, and a module-level skip, a skip marker or a conftest that
  skips is an error, because no test ran. A conftest raising Skipped
  at import no longer crashes the whole leela run: Skipped is a
  BaseException, outside what inner runs caught.
  0.8.0's verdict for a mutant that broke the target's import depended
  on the layout. With the virtualenv and the temp dir outside the
  project, it was SURVIVED with tests_run: 0. With the virtualenv
  inside the project, re-importing installed packages could crash the
  inner run, and the mutant was KILLED as <crashed>, a verdict
  indistinguishable from leela failing to test it.

• --leela-benchmark no longer raises TypeError. It called the
  target-discovery helpers without the python_files patterns they
  require.

• The django extra requires pytest-django>=4.10. 4.10 is the
  first release whose pytest_unconfigure restores the session's
  database block, which leela's nested sessions depend on.

• fail_on_error option in [tool.pytest-leela] (default true).
  Set it to false to keep errored mutants from failing the session.

v0.8.0

What's new

• Working-tree diff support in --diff mode. --diff HEAD now captures staged and unstaged edits via the union of the committed range (base...HEAD) and the working-tree/index diff (base, two-dot). Previously --diff HEAD against uncommitted edits produced an empty diff, 0 mutants, and a silent exit 0 — a hollow quality gate.

• Zero-mutant warning when --diff is active. When --diff is on and mutants_tested == 0, leela now emits a prominent WARNING so operators know the gate produced no signal rather than silently passing.

• Monorepo repo-root path normalization. Git-reported paths are now resolved against the real repo root rather than pytest's cwd, preventing double-subdir corruption in monorepos (e.g. services/api/services/api/x.py).

• Mutation-hardening tests for git_diff. All 41 mutations in git_diff.py are now killed (100%), up from 32/42 (76.2%).

v0.7.1 — Fix MutatingLoader not setting module.__file__

Fixed

MutatingLoader now populates __file__, __loader__, and __spec__ before executing mutated source. Previously, any target module that referenced __file__ at module scope (e.g. BASE_DIR = Path(__file__).resolve().parent in Django settings, asset path lookups via Path(__file__).parent / 'static' / ..., or pkg_resources.resource_filename(__name__, ...)) raised NameError: name '__file__' is not defined during the mutated import. The harness counted that import failure as "no test killed this mutant" and reported a false-positive SURVIVED, even when the existing tests would have caught the mutation.

The loader now mirrors the attribute population that importlib._bootstrap_external.SourceFileLoader performs via _init_module_attrs. Mutants that were silently absorbed as SURVIVED on __file__-using modules will now flip to KILLED — the tests were always correct; only the harness was reporting wrongly.

Downstream projects that introduced workarounds to dodge this bug (e.g. extracting __file__-touching code into helper modules, or routing asset lookups through framework finders solely to avoid the false positives) can revert those workarounds — they are no longer load-bearing for a green leela run.

Reported and root-caused in pith-task 0b048dd4-ac87-4e22-9c8b-789ae2b1bebb. Fixed in #10.

Full changelog: v0.7.0...v0.7.1

v0.7.0

What's Changed

• Add Python 3.14 to CI — now tested on both 3.13 and 3.14
• Remove bytecode mutation experiment — the experimental bytecode fast-path in import_hook never produced benchmark results and caused SIGSEGV when self-mutated on Python 3.14. Removed bytecode.py, its tests, and all supporting plumbing (-2,507 lines)
• Fix sched_setaffinity tests on macOS — tests now pass on all platforms
• Add Python 3.14 classifier to pyproject.toml

🤖 Generated with Claude Code

v0.6.0

What's Changed

Assignment-based dataflow type inference

pytest-leela now infers types through local variable assignments, even in code with no type annotations. This enables mutation pruning on previously untypeable code.

• Forward propagation: x = "hello" → x + y is recognized as string concatenation and pruned (0 mutations instead of 2)
• Parameter tracking: y = x where x: int correctly infers y as int
• Conflict detection: x = 1; x = "a" → type collapses to unknown (safe)
• Enrichment stats reporting: Terminal output now shows the breakdown:

  Type enrichment: 108 from annotations, 25 from assignment dataflow
  

Results on untyped code

• 25 additional type enrichments from dataflow on a zero-annotation module
• 7 additional mutations pruned
• ~0 extra time cost (pure AST walk, no external dependencies)

Also includes

• Extracted _type_of_value_node shared helper to deduplicate type inference
• EnrichmentStats dataclass with __add__ for multi-file aggregation
• 100% mutation kill rate (234/234 on target code, 590/590 overall)
• Realistic untyped test target module (assignment_targets.py)

Full Changelog: v0.5.0...v0.6.0

v0.5.0 — Respect pytest's python_files config for test exclusion

What's Changed

Bug Fix: Test file exclusion now respects python_files config

Previously, pytest-leela used hardcoded patterns (test_*.py, *_test.py) to exclude test files from mutation targets. This broke projects that use custom test file naming conventions, such as describe_*.py (common with pytest-describe).

Changes in this release:

• Test file exclusion now reads python_files from pytest.ini / pyproject.toml / setup.cfg instead of relying on hardcoded patterns
• Custom test file patterns (e.g., describe_*.py) are now correctly excluded from mutation targets when using --diff mode
• conftest.py is always excluded from mutation targets, regardless of configuration
• Pattern matching uses fnmatch for consistency with pytest's own behavior

Affected users: Anyone using --diff mode with a non-default python_files setting in their pytest config. Previously, files matching custom patterns would be included as mutation targets, which would generate false negatives (mutations in test infrastructure that don't kill any mutants).

No changes are required to your pytest config or test suite — pytest-leela now reads your existing config automatically.

Commits

• e730bab Fix --diff mode: exclude test files from mutation targets
• 36a6f68 Refactor test-file exclusion to use python_files config via fnmatch patterns
• 0c84183 Fix perfection game feedback: remove dead assertion, nest test block

v0.4.0

What's New

Per-Mutant Timeout

Control-flow mutations (break→continue, and→or in loops) can create infinite loops. Leela now sets an automatic timeout per mutant based on test execution times from the coverage pass:

• Timeout = max(2 × expected time + 1s, 5s)
• Timed-out mutants are marked as killed with killing_test="<timeout>"
• Cross-platform via threading.Timer

Bytecode Mutation Engine (Python 3.14+)

For BinOp, Compare, AugAssign, and UnaryOp mutations (~60% of all mutants), leela now patches compiled bytecode directly instead of going through ast.parse → visit → compile:

• Compile source once, produce mutant code objects via single-byte patches
• Dynamic opcode tables (portable across Python versions)
• Falls back to AST pipeline for complex mutations and Python < 3.14
• On Python 3.13 and below, all mutations use the AST pipeline (no behavior change)

Install / Upgrade

pip install pytest-leela==0.4.0

v0.3.0

What's New

Configurable Operators

• [tool.pytest-leela] config section with operators and exclude settings
• # leela: skip pragma to suppress mutations on specific lines
• Default target discovery: scans target/, src/, or flat layout automatically

Performance Optimizations

• Coverage sharing: line coverage is now collected during the normal test run (saves one full test suite execution per --leela invocation)
• sys.modules fast-clear: precomputed module sets for O(K) targeted clearing instead of O(N) full scan

Quality

• 100% mutation kill rate on self-mutation check (482/482 killed)
• Applied ruff format across codebase

Install / Upgrade

pip install pytest-leela==0.3.0

v0.2.0 — 5 New Mutation Operators

What's New

Five new mutation operator families, expanding pytest-leela's mutation coverage:

Bitwise Operators

• BitAnd (&), BitOr (|), BitXor (^), LShift (<<), RShift (>>)
• Includes type-aware pruning for int/bool operands

Augmented Assignments

• All 12 augmented assignment operators (+=, -=, *=, /=, //=, %=, **=, &=, |=, ^=, <<=, >>=)
• Mutations swap between compatible operator families

Inline Conditionals (Ternary)

• x if cond else y → y if cond else x (swap branches)
• x if cond else y → x (remove else)
• x if cond else y → y (remove if)

Break/Continue Swaps

• break → continue and continue → break inside loops

Exception Handler Mutations

• Broad exception handlers (except Exception) → narrow (except RuntimeError)
• Narrow handlers → broad handlers
• Equivalent mutant detection for single-exception handlers

Stats

• 386 tests passing
• Full self-mutation coverage
• Validated on real Django project (roaming-panda: 98.5%+ kill rate)

v0.1.0

Initial release of pytest-leela mutation testing plugin.

Features

• --leela pytest flag triggers mutation testing after test session
• AST-based mutation with type-aware pruning
• Coverage-guided test selection (only runs tests that cover mutated lines)
• Django framework cache clearing for accurate mutation testing
• --target flag for targeting specific files/directories
• --diff flag for testing only changed lines
• Exit code 1 when mutants survive (CI-friendly)
• Self-mutation verified: 100% kill rate on fixture targets