Problem
The collected test count is hard-coded, by hand, in six files: .bestpractices.json, CLAUDE.md, CONTRIBUTING.md, README.md, docs/ASSURANCE-CASE.md, assets/badge-tests.svg. Each is checked for exact equality against whichever branch's own live pytest --collect-only count runs in CI.
That count is a property of the post-merge tree, not of any one branch: two branches that each add tests compute two different, both-true numbers and must each edit the same six lines to match. Whichever merges second silently overwrites the first's correct figure with its own now-stale one.
Measured on this repo, not hypothesised:
Fix (this issue tracks the PR that resolves it)
assets/badge-tests.svg becomes the ONLY artifact stating an absolute test count.
- Its check moves from exact equality to a monotone floor (
committed <= live) in both scripts/check_doc_claims.py (doc_claim_structural.check_badge_floor) and scripts/generate_repo_badges.py (stale_tests_badge) — a committed count that lags the live one is stale-but-true and passes; only an over-claim fails.
- The other five files drop the number entirely, pointing at the badge instead.
- A standing regression test (
RepositoryTests.test_no_prose_file_states_the_suite_size_any_more) asserts no scanned file — .bestpractices.json included — states this claim in prose again.
scripts/check_doc_claims.py (420 lines) and scripts/generate_repo_badges.py (305 lines) were both over the repo's 300-line file cap before this work needed to touch them further; split via Extract Module into doc_claim_sources.py/doc_claim_scan.py/doc_claim_structural.py/repo_badge_catalog.py, verified behavior-preserving by an unchanged test suite before the floor logic was added.
Acceptance test: two throwaway branches off the fix branch, each adding a trivial passing test, merge with zero conflicts in the six files.
Problem
The collected test count is hard-coded, by hand, in six files:
.bestpractices.json,CLAUDE.md,CONTRIBUTING.md,README.md,docs/ASSURANCE-CASE.md,assets/badge-tests.svg. Each is checked for exact equality against whichever branch's own livepytest --collect-onlycount runs in CI.That count is a property of the post-merge tree, not of any one branch: two branches that each add tests compute two different, both-true numbers and must each edit the same six lines to match. Whichever merges second silently overwrites the first's correct figure with its own now-stale one.
Measured on this repo, not hypothesised:
mainwent red twice between merges purely from this race (PR fix(mutmut): dot the last scripts/ test-loader gap, make pip_constraint_sets.py mutation-testable (#262) #280 synced the number to its own total; fix: drain pending tasks before stopping the AP sync loop (issue #258) #278 then added more tests against a now-stale base).Fix (this issue tracks the PR that resolves it)
assets/badge-tests.svgbecomes the ONLY artifact stating an absolute test count.committed <= live) in bothscripts/check_doc_claims.py(doc_claim_structural.check_badge_floor) andscripts/generate_repo_badges.py(stale_tests_badge) — a committed count that lags the live one is stale-but-true and passes; only an over-claim fails.RepositoryTests.test_no_prose_file_states_the_suite_size_any_more) asserts no scanned file —.bestpractices.jsonincluded — states this claim in prose again.scripts/check_doc_claims.py(420 lines) andscripts/generate_repo_badges.py(305 lines) were both over the repo's 300-line file cap before this work needed to touch them further; split via Extract Module intodoc_claim_sources.py/doc_claim_scan.py/doc_claim_structural.py/repo_badge_catalog.py, verified behavior-preserving by an unchanged test suite before the floor logic was added.Acceptance test: two throwaway branches off the fix branch, each adding a trivial passing test, merge with zero conflicts in the six files.