The mutation-testing protocol is undocumented, and three of its failure modes produce false greens

[realmarcin]

Summary

This repository's gate tests are written and defended by mutation testing —
"revert the fix and confirm the specific test goes red." A dozen modules carry
an explicit test_the_check_can_actually_fail. But the protocol for running a
mutation check is nowhere written down, and it has three failure modes that each
produce a confident, wrong conclusion. All three were hit in a single session.

The three ways a mutation check lies

1. The mutation never applied. A replace() targeting text that Black had
reflowed, or a string appearing twice, silently changes nothing — and the green
run is read as "the test is fine" when it means "nothing was tested."

Observed twice: a _BLIND tuple entry that Black had collapsed to one line,
and parents[2] in src/communitymech/paths.py, which appears twice so the
single-occurrence assumption failed.

2. Stale bytecode served the wrong module. Fixed mechanically in #693, but
worth stating in the protocol because the fix is invisible to someone reading a
test.

3. The mutation was real but undetectable on this machine. Changing
kb/taxa to kb/TAXA is a genuine behaviour change on Linux and a no-op on
macOS, where the filesystem ignores case. The check goes green and says nothing.

Note the asymmetry: (1) and (3) produce false greens — a test certified as
able to fail when it cannot. That is the direction that matters, because the
whole point of the ritual is to catch a test that cannot fail.

Proposed

A short, checkable protocol, written once where the discipline is stated
(CLAUDE.md, or a tests/README.md that the gate modules can reference):

1. Prove the mutation applied before trusting a red — grep -c the new
   text, or assert the replacement count. A red from an unapplied mutation is
   impossible, so this only ever catches false greens, which is the point.
2. Prove the restore took — the suite must be green on the very next run,
   not one run later. A second run being needed is itself the Tests loading scripts/ via spec_from_file_location can run stale bytecode, which can certify a test that cannot fail #693 symptom.
3. Choose a mutation the test can see on this machine. Prefer changing a
   value the assertion names over changing a path, a filename, or anything the
   filesystem may normalise.
4. Back up by copy, not by git checkout --. Restoring with git checkout
   discards unrelated uncommitted work in the same file; cp to a scratch path
   and copy back does not.

Why a document rather than a tool

A helper that performs mutations would be nice and is a bigger change; it also
cannot choose a semantically meaningful mutation, which is step 3 and the part
requiring judgement. Writing the protocol down is cheap and addresses the
observed failures directly. A tool can come later if the protocol proves
insufficient.

Acceptance test

The protocol exists in a discoverable place, and a gate module's
test_the_check_can_actually_fail docstring can cite it instead of
re-explaining the ritual.