fix(mutmut): dot the last scripts/ test-loader gap, make pip_constraint_sets.py mutation-testable (#262) - #280
fix(mutmut): dot the last scripts/ test-loader gap, make pip_constraint_sets.py mutation-testable (#262)#280cdeust wants to merge 2 commits into
Conversation
…raint_sets.py mutation-testable (#262) test_typecheck_env_parity.py loaded scripts/pip_constraint_sets.py under the bare name "pip_constraint_sets" instead of the path-derived "scripts.pip_constraint_sets" mutmut keys mutant trampolines on — #264 fixed this for 14 sibling scripts/ test files but missed this one, so a scoped mutation run on it aborted with "Stopping early ... none match any mutant key" (found by code-reviewer post-merge on #264). Dotting the loader alone was not sufficient: ConstraintSet's three methods (path/command/header_lines) carried its entire logic, and mutmut's mutation generator categorically excludes the body of any @dataclass-decorated class (confirmed empirically: 0 mutants for the whole file with the methods in place, vs 58 after extracting them to free functions constraint_path/constraint_command/constraint_header_lines with ConstraintSet left as pure data). Every call site across generate_pip_constraints.py and its two test files is updated to match; the existing 301-test tests_py/scripts/ suite passes unchanged, proving the refactor is behavior-preserving. A dedicated tests_py/scripts/test_pip_constraint_sets.py directly exercises the three extracted functions (neither existing test file called them for real: everything went through compose()/render(), which every test either mocks or short-circuits before reaching). The scoped mutation run (tests_py/scripts/test_typecheck_env_parity.py + test_pip_constraint_sets.py against scripts/pip_constraint_sets.py) now produces real attribution: 58 mutants, 58 killed, 0 survived, 0 no-tests. Boy-scout finds in the same sweep (grepped every spec_from_file_location and sys.path-plus-bare-import in tests_py/, 21 call sites verified): - tests_py/scripts/test_generate_repo_badges.py bare-imported scripts/badge_render.py (`import badge_render` after a sys.path insert) instead of loading it under "scripts.badge_render" — same defect class, now fixed with its own dotted load, independent of generate_repo_badges.py's own bare import of the same file. - Two ruff E501 violations introduced by this diff's own line-length growth, fixed in the same commit (§14). - generate_repo_badges.py's RepoBadge class has the identical @dataclass-exclusion gap (confirmed: 234 mutants total, 0 attributed to RepoBadge.path()/.render()) — out of this PR's blast radius (a separate root-cause fix of comparable size; mixing it in here would violate §15.1's "don't mix the enabling refactor with the fix"); reported with evidence rather than filed, per this task's no-new- issues instruction. Resynced the 6 suite-count sites + badge SVG to the measured absolute (6560, +10 for test_pip_constraint_sets.py) via generate_repo_badges.py and manual doc edits; both check_doc_claims.py --test-count 6560 and generate_repo_badges.py --check --test-count 6560 pass. Full suite: 6560 passed, 117 subtests passed (167.83s). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u
…onstraint_sets.py scoped runs tests_py/scripts/test_typecheck_env_parity.py reads .github/workflows/ci.yml directly (cross-checking CONTRIBUTING.md's documented type-check command against it). Running scripts/mutation_check.sh with this test in the selection failed collection under every mutant with FileNotFoundError (mutants/.github/workflows/ci.yml missing) instead of scoring the suite -- found while producing the real mutant tally for the previous commit's pip_constraint_sets.py fix. .github wasn't in also_copy for any prior scoped run (json_native.py, launcher_deps.py, generate_pip_constraints.py all read files already listed), so this is additive and does not change behavior for any other scoped run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u
|
Discharging the boy-scout deferral flagged in review. The Branch: That is a stronger discharge of §14 than an issue number would be: the defect gets fixed, not tracked. Also noted from review, non-blocking: the sweep tally in this PR's body should read 22 sites (15 + 7), not 21 (15 + 6) — |
Summary
tests_py/scripts/test_typecheck_env_parity.pystill loadedscripts/pip_constraint_sets.pyunder the bare synthetic name"pip_constraint_sets"instead of the path-derived"scripts.pip_constraint_sets"mutmut keys mutant trampolines on. #264 fixed this exact defect for 14
sibling
scripts/test files but missed this one (found by code-reviewerpost-merge on #264, filed conceptually under issue #262).
Dotting the loader name alone was not sufficient to produce a real
mutant tally:
ConstraintSet's three methods (path/command/header_lines)carried 100% of the file's logic, and mutmut's mutation generator
categorically excludes the body of any
@dataclass-decorated class (itmust — copying a decorated class for the trampoline setup can re-run the
decorator's side effects). Confirmed empirically: 0 mutants for the
whole file with the logic as methods, vs 58 mutants after extracting
constraint_path/constraint_command/constraint_header_linesintofree functions and leaving
ConstraintSetas pure data. Every call siteacross
generate_pip_constraints.pyand its two test files is updated tomatch; the pre-existing 301-test
tests_py/scripts/suite passesunchanged, proving the refactor is behavior-preserving.
What changed
tests_py/scripts/test_typecheck_env_parity.py— dotted thepip_constraint_setsloader (the reported gap).scripts/pip_constraint_sets.py— extractedConstraintSet's threemethods to module-level functions (data/behavior split), so the logic is
no longer hidden inside a
@dataclass-excluded class body.scripts/generate_pip_constraints.py+tests_py/scripts/test_generate_pip_constraints.py— updated every call site (
.path()→constraint_path(...),.command()→
constraint_command(...),.header_lines()→constraint_header_lines(...)).tests_py/scripts/test_pip_constraint_sets.py(new) — a dedicated,direct unit-test file for the three extracted functions. Neither existing
test file called them for real: every prior real invocation went through
compose()/render(), which every existing test either mocks orshort-circuits before reaching (this is what "0 killed, 31 no-tests, 22
survived" surfaced on the first honest run, before this file existed).
tests_py/scripts/test_generate_repo_badges.py— boy-scout find fromthe directory-wide sweep: it bare-imported
scripts/badge_render.py(
import badge_renderafter insertingscripts/ontosys.path)instead of loading it under
"scripts.badge_render"— same defect class.Fixed with its own dotted load, independent of
generate_repo_badges.py'sown (unaffected, still-bare) internal import of the same file.
pyproject.toml—.githubwas missing from[tool.mutmut] also_copy;test_typecheck_env_parity.pyreads.github/workflows/ci.ymldirectly,so any scoped mutation run selecting it failed collection under every
mutant with
FileNotFoundErrorinstead of scoring the suite. Found whileproducing the tally below; purely additive, no effect on any other
scoped run (json_native.py/launcher_deps.py/generate_pip_constraints.py
all read files already listed).
ruffE501 violations this diff's own line growth introduced,fixed in the same commit (§14).
six hard-coded sites + the committed badge SVG, regenerated via
generate_repo_badges.py --test-count 6560and verified withcheck_doc_claims.py --test-count 6560.Directory-wide sweep (deliverable #4)
grep -rn "spec_from_file_location\|sys.path.insert\|sys.path.append" tests_py/— 21 call sites verified by hand:
spec_from_file_locationsites: all 13 already-fixedscripts/testfiles confirmed still dotted correctly; the 1 reported gap (fixed above);
1 more found and fixed (
badge_render, above).tests_py/fuzz/test_corpus_replay.pyand
tests_py/infrastructure/test_config.pyalso use this pattern butload files outside the current committed mutmut
source_paths = ["mcp_server"]scope and outside issue Mutation testing cannot attribute mutants for any scripts/ module (synthetic module names in tests_py/scripts) #262'sscripts/blast radius —test_config.py's reload is a deliberate one-off isolation instancealongside ~40 normal dotted importers of
mcp_server.infrastructure.configthat already provide correct attribution, so it is not this defect.
sys.path.insert/appendsites: 3 wiki-migration test files already usethe correct
from scripts.X import ...dotted form (citing issue Mutation testing cannot attribute mutants for any scripts/ module (synthetic module names in tests_py/scripts) #262 intheir own comments); the other 3 (
test_goal_maintenance.py,test_goal_maintenance_wiring.py,benchmark_harness.py) insert"."only to make
mcp_serverimportable and already import it dotted(
from mcp_server.core import ...) — not an instance of this defect.Mutation tally (real, quoted — deliverable #2/#3)
Reproducing the reported abort, literal invocation exactly as this
task named the files:
(against the pre-fix tree, bare loader name) —
After dotting the loader name alone (methods still on
ConstraintSet,before the data/behavior split):
mutate_file_contents()on the filereturns 0 mutants — confirmed via direct call, independent of any
test — because mutmut's
MutationVisitor._skip_node_and_childrencategorically excludes
@dataclass-decoratedClassDefbodies.After the full fix, same literal 2-arg invocation (
scripts/mutation_check.sh tests_py/scripts/test_typecheck_env_parity.py scripts/pip_constraint_sets.py,scoping the test selection to exactly the ONE file this task named — the
script's CLI takes a single test path):
Real attribution now happens (no more abort) — but that ONE test file
alone never called
constraint_command/constraint_header_lineswithenough assertions to kill everything, because
pip_constraint_sets.pyisalso exercised by
test_generate_pip_constraints.pyand the new dedicatedtest_pip_constraint_sets.py, neither of which the script's single-test-pathCLI can express alongside the other in one invocation.
Full, honest triage — scoping
pytest_add_cli_args_test_selectiontoBOTH
test_typecheck_env_parity.pyand the newtest_pip_constraint_sets.py(mirroring
scripts/mutation_check.sh's ownpyproject.tomlpatchinglogic by hand, then invoking
mutmut rundirectly against the shared venvto avoid
uv run's fresh-venv side effect):58 mutants, 58 killed, 0 survived, 0 no-tests. Every one of the 53
non-trivially-killed mutants was a missing test, now added in
test_pip_constraint_sets.py— none were equivalent; every mutationobserved was a real, distinguishable behavior change (a wrong CLI flag, a
dropped accumulator, a corrupted banner line).
Boy-scout defect found, evidenced, NOT fixed (too large for this PR)
generate_repo_badges.py'sRepoBadgeclass has the identical@dataclass-exclusion gap: confirmed empirically, 234 total mutants forthe file, 0 attributed to
RepoBadge.path()/.render(). Fixing itrequires the same shape of work as this PR (extract to free functions,
update every call site in
generate_repo_badges.py+ its test file, adddirect tests, re-run mutation testing) — a second, independent root-cause
fix of comparable size. Mixing it into this PR would violate §15.1 ("don't
mix the enabling refactor with the fix") and roughly double this diff's
blast radius beyond its named subject (
pip_constraint_sets.py/ issue#262). Per this task's explicit no-new-issues instruction, reported here
with evidence rather than filed.
Test plan
pytest tests_py/scripts/— 311 passed, 117 subtests passed(was 301 before this PR's new test file)
pytest -q— 6560 passed, 117 subtests passed (167.83s)python scripts/generate_pip_constraints.py --check—requirements OK (13 checked)python scripts/check_doc_claims.py --test-count 6560— OKpython scripts/generate_repo_badges.py --check --test-count 6560— OKruff check/ruff format --checkon every touched file — cleanscripts/mutation_check.shrun twice (before/after) + a manually-scopedfull-triage run — real tallies quoted above, 0 survivors in the final state
(
generate_pip_constraints.py'sfrom pip_constraint_sets import ...,generate_repo_badges.py/refresh_mcp_toplist_badge.py'simport badge_render) verified to still resolve via real subprocessinvocation, unaffected by the test-loader renames.
🤖 Generated with Claude Code
https://claude.ai/code/session_012Yu6EnWspTfqHoGkExyS6u