Part 1: Run-cohort safety (run_id, purge command, staleness guard)

[WilfordGrimley]

Summary

Part 1 of the six-part catalog-completion plan tracked in docs/features/catalog-completion-plan.md — makes the owner's aggressive stop → merge → rebuild → restart iteration style safe. HOLD #A per the plan: held for review before Parts 2/3 proceed, and before the live full-catalog run is next restarted with this code.

• Schema: run_id added to AbstractWeightedVote (covers CardPrintingTag/CardArtistVote/CardTagVote in one migration) — a separate field, not an anonymous_id suffix. Confirmed via direct investigation (two research passes) that anonymous_id's exact-match reuse across invocations is load-bearing for the existing idempotence/resume mechanism, and its max_length=40 would hard-block a stamped value for at least two engine names anyway. New PilotRunLedger model for the durable, queryable run record.
• Threading: every machine vote-casting site in local_identify_printing_tags.py/local_fallback.py now stamps its vote with the invoking run's run_id. Human-submitted votes are untouched (stay NULL).
• Staleness guard: find_stale_applied_migrations — compares what this image's own migrations know about against what the DB reports as applied, refuses to start if the image is older than a previously-deployed one. Automates the PR Pre-scale program: source exclusion, yield reconciliation, future-work note #24/Two pilot fast-follows: expansion_hint narrowing, name-frequency elimination #26 stale-image lesson instead of relying on manually checking docker images timestamps. Caught a real bug via its own test (MigrationRecorder.applied_migrations() returns a dict, not a set — the first draft would have raised TypeError on first real use).
• Git-SHA baking: best-effort visibility only (Dockerfile ARG + docker-compose build.args) — logged at startup, never itself the gate. Rebuild command now requires a GIT_SHA=$(git rev-parse --short HEAD) prefix (documented).
• purge_machine_votes --run-id <id> [--dry-run]: deletes exactly one invocation's votes, re-resolves every affected card via the persisting consensus resolvers.

The post-purge invariant, stated precisely

"Corrected" isn't reviewable without the exact statement, so here it is verbatim against the implementation (purge_machine_votes.verify_no_machine_only_resolutions):

After a real (non-dry-run) purge, every affected card is re-resolved from scratch via the persisting consensus resolvers (resolve_and_persist_printing/resolve_and_persist_artist/resolve_and_persist_tag_votes), using whatever votes actually remain — not a diff against pre-purge state. The command then asserts: for every affected card whose printing_tag_status is RESOLVED, at least one surviving CardPrintingTag vote for that resolved printing has a human-backed source (not VoteSource.DEDUCTION/VoteSource.OCR); identically for artist_vote_status against the resolved artist, and per-tag for tag_vote_statuses entries that are RESOLVED_APPLY/RESOLVED_REJECT.

A card is NOT required to return to its pre-purge status — un-resolving as a consequence of losing machine-only weight is the expected, correct outcome, reported separately (cards_unresolved_by_purge), never a violation. Only a RESOLVED outcome with zero surviving human-backed votes behind it is a violation (gate_violations, raises CommandError) — resolve_weighted_consensus's own human-backed gate should make that structurally impossible, so if it ever fires it means something upstream broke, not that the purge did anything wrong.

(With the real default weights — PRINTING_TAG_MIN_VOTES=2, PRINTING_TAG_AI_WEIGHT=0.5, human weight 1.0 — 1 human vote + 2 agreeing machine votes sums to 2.0 and resolves; purging those 2 machine votes correctly drops the weight below threshold and legitimately un-resolves the card. The original task framing, "assert statuses return to pre-run state," would have false-positived on exactly this case.)

Merge + migrate sequencing with the live run in flight

The migration is additive-only (AddField(run_id, null=True, db_index=True) on three small tables — max 41,571 rows, confirmed via pg_stat_user_tables) — a metadata-only ALTER TABLE in Postgres 11+, plus a near-instant CREATE INDEX given the column starts 100% NULL. Safe to apply live under the running job's concurrent writes.

The running job keeps writing votes with run_id NULL for the rest of its current invocation once this merges and migrates — that's correct and meaningful, not a gap. NULL run_id identifies the pre-safety-era cohort: still fully governed by the never-resolve gate and restart-safety (neither was ever conditional on run_id existing), just not individually purgeable by run the way anything stamped going forward is. The run is not restarted solely to gain stamping — it folds in naturally at whatever iteration the run next gets restarted for anyway. See the "Cohort convention" note in docs/features/printing-tags.md.

Expected side effect: once this merges and migrates, any restart from an older (pre-migration) image trips the staleness guard and refuses to start, forcing a rebuild first. That's the guard doing its job, not a bug.

• Docs: new "Iteration safety" section in docs/features/printing-tags.md stating the complete safety-property set (2 pre-existing: never-resolve gate, restart-safety; 2 new: revocability, staleness guard), the precise post-purge invariant, and the NULL-run_id cohort convention.

Test plan

• pytest cardpicker/tests/ — 756 passed (plus the same pre-existing, unrelated moxfield failures documented in task Document Part 4's missing batch-flush checkpointing #122)
• pre-commit run (ruff, isort, black, mypy, prettier) — all clean
• makemigrations --check — migration matches model state exactly
• New tests: run_id threading (5 tests), find_stale_applied_migrations/get_baked_git_sha (5 tests), purge_machine_votes (11 tests covering dry-run, real purge with the corrected invariant, multi-run isolation, orphaned-ledger tolerance, and a deliberately-constructed "impossible" gate-violation state), PilotRunLedger lifecycle (4 tests: dry-run creates nothing, real run completes, mid-run exception fails cleanly not danglingly, staleness refusal creates nothing)
• Hit and fixed the documented shared-factory-sequence-counter issue (docs/lessons.md) in the new test file before it could silently break unrelated test_views.py snapshots

Does not touch the live full-catalog run (branched from master, independent of it).

🤖 Generated with Claude Code

https://claude.ai/code/session_016i9S7LQsCL3FGaih3ZTRBJ