Skip to content

Add mutation testing for the state machine and engine #103

Description

@bagowix

Motivation

Coverage is enforced at 100% branch coverage (fail_under = 100), which proves
every line and branch executes — not that any assertion would notice if the
behaviour changed. For a state machine whose whole value is threshold arithmetic
and transition ordering, that gap matters: flipping >= to >, swapping a
threshold comparison, or dropping a state reset can easily leave a
100%-covered suite green.

Mutation testing closes that gap and produces a number worth publishing next to
the coverage badge.

Proposal

Add mutmut (3.x) as a dev-only tool, scoped to the modules where a surviving
mutant is a real bug:

  • interlock/_state_machine.py (257 lines) — transitions, thresholds, probe
    admission;
  • interlock/_engine.py (562 lines) — lock scope, dispatch, recording order.

Run it out of band, not on every PR: mutation runs are minutes-to-tens-of-
minutes, and the signal is a slowly-moving score, not a per-commit gate.

Acceptance criteria

  • mutmut configured in pyproject.toml with the two modules above as the
    target and the fast subset of tests as the runner (exclude the
    wall-clock tests/test_examples.py and the extras integration tests).
  • A documented local command in CONTRIBUTING.md.
  • A scheduled (weekly) or manually dispatched workflow that runs mutation
    testing and surfaces the score; not a required PR check.
  • Baseline score recorded in the issue or the repository so later runs have a
    comparison point.
  • Every surviving mutant is either killed by a new test or documented as
    equivalent/intentional.
  • The core stays dependency-free.

Non-goals

  • Extending mutation testing to the integrations (interlock/integrations/) or
    the pipeline in the first pass.
  • A hard mutation-score threshold as a merge gate.
  • Replacing the coverage gate — the two are complementary.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions