Per-face illustration ids; delete the border-colour "multi-faced" gate; stage-d-illustration-v2 - #565
Merged
Merged
Conversation
…e; bump to stage-d-illustration-v2 `stage-d-illustration-v1` cast 3 CardIllustrationVote rows in its entire existence against 230,753 catalog cards. Its gate skipped every card whose `ImageEvidence.layout_class` was non-blank, on the stated premise that the column records faced-ness. It does not: its only writer is `local_fallback.classify_border_color` and it holds a BORDER COLOUR (live: black 138,728 / borderless 72,603 / white 7,475 / '' 1,455 / silver 408 - non-blank on 99.34% of rows). The gate discarded 99.28% of every population handed to the calculator; 3,409 of its 3,426 scanned rows logged `multi-faced-v1`. 1. `CanonicalPrintingMetadata.face_illustrations` (JSONField, migration 0095) retains EVERY face's own `illustration_id` paired with that face's name, in `card_faces` order. The scalar `illustration_id` still holds the FRONT face for the four consumers that read it. A partial index (`cpm_face_illustrations_present`) backs the new version-stamp term. 2. Populated only for `DOUBLE_FACED_LAYOUTS` - split/adventure/flip/aftermath/ mutate/prototype nest MODES on one printed face and gain no phantom entry. 3. The single-faced gate is DELETED, not repaired: `IllustrationIndex` now files each face's artwork under that face's own name, and `_resolve_illustration_candidates` widens a back-face-named upload to its combined DFC name for CANDIDATES while keying the ILLUSTRATION lookup on the back-face name - so a back-face scan resolves to the artwork actually printed on the side that was scanned. `printings_for_illustration` widens to match face ids too. 4. `test_skips_multi_faced_cards` (fixture `layout_class="split"` - a value the field's only writer can never emit) is replaced by tests parametrised over the field's REAL vocabulary, per-face index tests, and a back-face-named upload end-to-end. Eight independent un-patch mutations were verified RED. 5. Bumped to `stage-d-illustration-v2`: `multi-faced-v1` is not in RESCANNABLE_SKIP_REASONS and `_eligible_illustration_cards_queryset` excludes cards with a non-rescannable scan log for its own anonymous_id, so a repaired v1 would never re-examine the 3,409 cards it wrongly skipped. `models.calculator_family` strips `-vN`, so purge/dedupe/zero-weight behaviour follows automatically. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
PR #562 landed a CI tether requiring every `*_ANONYMOUS_ID` declared in `MPCAutofill/cardpicker/` to have an entry in docs/pipeline-fidelity-gate.md, keyed on the CURRENT identity including its version suffix. Updates that roster entry from `-v1` ("DORMANT, under repair... a fix is in flight on a separate branch" — this branch) to `-v2`, recording what the repair was and that nothing has been measured in prod yet, and moves the derivation guard test's hardcoded identity with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
WilfordGrimley
force-pushed
the
feat/per-face-illustration-ids
branch
from
July 29, 2026 11:37
b46f32c to
fb45613
Compare
… figures The first replay used a 2,500-card sample, which put 39 printing verdicts against ~193 considered cards - too few to separate signal from binomial noise (the projected 2,505 sat ~2.9 sigma below the brief's own ~3,610 estimate). Re-run at 30,000: 2,350 considered, 1,920 illustration verdicts, 604 printing verdicts, projecting ~10,277 / ~3,233 catalog-wide. Both now sit within 10% of the independently-derived expectation, so the earlier printing shortfall was sampling error, not a defect in the fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
WilfordGrimley
added a commit
that referenced
this pull request
Jul 29, 2026
…ation from the first line `CardIllustrationVote` (issue #524) has been WRITTEN since it landed and read by nothing but the admin and the human write path. `printing_consensus.py`, `artist_consensus.py` and `tag_consensus.py` each reconcile their own vote model; there was no illustration equivalent, so PR #565's projected ~10,277 machine rows would have been recorded rather than reasoned over. 1. `cardpicker/illustration_consensus.py`, modelled on its two siblings and built on the shared `vote_consensus` core - no weighting, quorum or human-backed gate is re-derived. `is_unknown` is tallied as an ORDINARY OUTCOME KEY (the `UNKNOWN` sentinel, exactly as `artist_consensus` treats `CardArtistVote.is_unknown`): the abstention on this model is the ABSENCE OF A ROW, so an `is_unknown=True` row is a positive claim that must be able to resolve on its own AND to contest a uuid. 2. MD5 POOLING IS BUILT IN, NOT RETROFITTED. Every read path is group-scoped: `resolve_illustration` tallies `card`'s whole md5 identity group, pooled per agent by `vote_consensus.pool_group_votes`, keyed on `agent_dedupe_key` (the versionless calculator family - #565 bumping this calculator v1->v2 is exactly the case a raw-id key would misread as two agents). The group primitives are IMPORTED from `printing_consensus`, so there is one definition of a group, one completeness guard and one agent-identity rule. md5 is strictly stronger than the perceptual art hash here: byte-identical files are necessarily the same artwork, with no threshold and no tolerance. 3. PROPAGATION IS A CONSEQUENCE OF GROUP-SCOPED RESOLUTION, NOT A SEPARATE STEP. A member whose decorated name fails candidate resolution abstains with `no-candidate-match` (367 of 2,350 considered cards in #565's replay) even though a byte-identical sibling resolved. Its tally IS the group's tally, and `resolve_and_persist_illustration` writes the outcome to every member, so it receives the resolved artwork with no path aware it abstained. The rejected alternative - writing a copied `CardIllustrationVote` row - would contribute exactly ZERO weight (it pools under the same agent key; pinned by a test), would manufacture a claim no agent made, and would collide with the model's unconditional (card, anonymous_id) constraint. `Card.inferred_illustration_id` (plain UUIDField, not an FK - no `CanonicalIllustration` table, and no reference-data join to go stale) plus `Card.illustration_vote_status` (migration 0096), written for every group member. `ILLUSTRATION_MIN_VOTES`/`ILLUSTRATION_MIN_SHARE` default to the printing values, changing nothing; there is deliberately no illustration machine weight, since a vote's weight is a property of who cast it, never of what is being voted on. REFERENCE DATA (owner ruling 2026-07-29): this module reads neither `CanonicalCard` nor `CanonicalPrintingMetadata` - it tallies uuids off vote rows and stores the winner verbatim, asserted by `TestReferenceDataIndependence`. A stale snapshot can under-supply the uuids agents have to vote for (upstream) and narrow what a resolved uuid maps to (downstream, a live join every consumer performs itself); it cannot change a tally, flip a winner, or move a propagation. TESTS. 50 new tests, and every one was verified RED against a deliberately mutated implementation (24 mutations; each of the 50 fails under at least one). Coverage includes md5 pooling (agreeing siblings collapse; a self-contradicting agent is withheld order-independently; distinct agents still sum; a group of one is a byte-for-byte no-op including query shape), propagation (a voteless sibling inherits; it does NOT inherit across an identical `content_phash`, with a same-phash-plus-matching-md5 positive control proving the negative is not vacuous), and the human-backed gate for this vote type. HARNESS FIX, and it is not incidental. `test_shared_cache.py::TestSharedCacheTable Migration` carries `@pytest.mark.django_db(transaction=True)`, so its migration round-trip REALLY COMMITS - and its `finally` restored only as far as `0092`, leaving every later migration unapplied for the rest of the session. Latent and free until now purely by alphabetical luck (0093/0094 are data-only; 0095 is read only by modules sorting earlier). A migration adding a `Card` column is the first thing to step on it: `test_sources.py` and `test_stage_e_dispatch.py` sort later and use `transactional_db`, and failed with `column "inferred_illustration_id" ... does not exist` - a message pointing at this branch for a defect entirely in that `finally`. Now restores to `graph.leaf_nodes("cardpicker")`, derived rather than hardcoded. VERIFIED: full `cardpicker/tests/` suite green (3,090 passed, 9 skipped) against a 3,040-passed baseline on the same commit of master. black/isort/ruff/mypy and docs-lint clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
WilfordGrimley
added a commit
that referenced
this pull request
Jul 29, 2026
…ation from the first line `CardIllustrationVote` (issue #524) has been WRITTEN since it landed and read by nothing but the admin and the human write path. `printing_consensus.py`, `artist_consensus.py` and `tag_consensus.py` each reconcile their own vote model; there was no illustration equivalent, so PR #565's projected ~10,277 machine rows would have been recorded rather than reasoned over. 1. `cardpicker/illustration_consensus.py`, modelled on its two siblings and built on the shared `vote_consensus` core - no weighting, quorum or human-backed gate is re-derived. `is_unknown` is tallied as an ORDINARY OUTCOME KEY (the `UNKNOWN` sentinel, exactly as `artist_consensus` treats `CardArtistVote.is_unknown`): the abstention on this model is the ABSENCE OF A ROW, so an `is_unknown=True` row is a positive claim that must be able to resolve on its own AND to contest a uuid. 2. MD5 POOLING IS BUILT IN, NOT RETROFITTED. Every read path is group-scoped: `resolve_illustration` tallies `card`'s whole md5 identity group, pooled per agent by `vote_consensus.pool_group_votes`, keyed on `agent_dedupe_key` (the versionless calculator family - #565 bumping this calculator v1->v2 is exactly the case a raw-id key would misread as two agents). The group primitives are IMPORTED from `printing_consensus`, so there is one definition of a group, one completeness guard and one agent-identity rule. md5 is strictly stronger than the perceptual art hash here: byte-identical files are necessarily the same artwork, with no threshold and no tolerance. 3. PROPAGATION IS A CONSEQUENCE OF GROUP-SCOPED RESOLUTION, NOT A SEPARATE STEP. A member whose decorated name fails candidate resolution abstains with `no-candidate-match` (367 of 2,350 considered cards in #565's replay) even though a byte-identical sibling resolved. Its tally IS the group's tally, and `resolve_and_persist_illustration` writes the outcome to every member, so it receives the resolved artwork with no path aware it abstained. The rejected alternative - writing a copied `CardIllustrationVote` row - would contribute exactly ZERO weight (it pools under the same agent key; pinned by a test), would manufacture a claim no agent made, and would collide with the model's unconditional (card, anonymous_id) constraint. `Card.inferred_illustration_id` (plain UUIDField, not an FK - no `CanonicalIllustration` table, and no reference-data join to go stale) plus `Card.illustration_vote_status` (migration 0096), written for every group member. `ILLUSTRATION_MIN_VOTES`/`ILLUSTRATION_MIN_SHARE` default to the printing values, changing nothing; there is deliberately no illustration machine weight, since a vote's weight is a property of who cast it, never of what is being voted on. REFERENCE DATA (owner ruling 2026-07-29): this module reads neither `CanonicalCard` nor `CanonicalPrintingMetadata` - it tallies uuids off vote rows and stores the winner verbatim, asserted by `TestReferenceDataIndependence`. A stale snapshot can under-supply the uuids agents have to vote for (upstream) and narrow what a resolved uuid maps to (downstream, a live join every consumer performs itself); it cannot change a tally, flip a winner, or move a propagation. TESTS. 50 new tests, and every one was verified RED against a deliberately mutated implementation (24 mutations; each of the 50 fails under at least one). Coverage includes md5 pooling (agreeing siblings collapse; a self-contradicting agent is withheld order-independently; distinct agents still sum; a group of one is a byte-for-byte no-op including query shape), propagation (a voteless sibling inherits; it does NOT inherit across an identical `content_phash`, with a same-phash-plus-matching-md5 positive control proving the negative is not vacuous), and the human-backed gate for this vote type. HARNESS FIX, and it is not incidental. `test_shared_cache.py::TestSharedCacheTable Migration` carries `@pytest.mark.django_db(transaction=True)`, so its migration round-trip REALLY COMMITS - and its `finally` restored only as far as `0092`, leaving every later migration unapplied for the rest of the session. Latent and free until now purely by alphabetical luck (0093/0094 are data-only; 0095 is read only by modules sorting earlier). A migration adding a `Card` column is the first thing to step on it: `test_sources.py` and `test_stage_e_dispatch.py` sort later and use `transactional_db`, and failed with `column "inferred_illustration_id" ... does not exist` - a message pointing at this branch for a defect entirely in that `finally`. Now restores to `graph.leaf_nodes("cardpicker")`, derived rather than hardcoded. VERIFIED: full `cardpicker/tests/` suite green (3,090 passed, 9 skipped) against a 3,040-passed baseline on the same commit of master. black/isort/ruff/mypy and docs-lint clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
WilfordGrimley
added a commit
that referenced
this pull request
Jul 29, 2026
…ation from the first line (#573) `CardIllustrationVote` (issue #524) has been WRITTEN since it landed and read by nothing but the admin and the human write path. `printing_consensus.py`, `artist_consensus.py` and `tag_consensus.py` each reconcile their own vote model; there was no illustration equivalent, so PR #565's projected ~10,277 machine rows would have been recorded rather than reasoned over. 1. `cardpicker/illustration_consensus.py`, modelled on its two siblings and built on the shared `vote_consensus` core - no weighting, quorum or human-backed gate is re-derived. `is_unknown` is tallied as an ORDINARY OUTCOME KEY (the `UNKNOWN` sentinel, exactly as `artist_consensus` treats `CardArtistVote.is_unknown`): the abstention on this model is the ABSENCE OF A ROW, so an `is_unknown=True` row is a positive claim that must be able to resolve on its own AND to contest a uuid. 2. MD5 POOLING IS BUILT IN, NOT RETROFITTED. Every read path is group-scoped: `resolve_illustration` tallies `card`'s whole md5 identity group, pooled per agent by `vote_consensus.pool_group_votes`, keyed on `agent_dedupe_key` (the versionless calculator family - #565 bumping this calculator v1->v2 is exactly the case a raw-id key would misread as two agents). The group primitives are IMPORTED from `printing_consensus`, so there is one definition of a group, one completeness guard and one agent-identity rule. md5 is strictly stronger than the perceptual art hash here: byte-identical files are necessarily the same artwork, with no threshold and no tolerance. 3. PROPAGATION IS A CONSEQUENCE OF GROUP-SCOPED RESOLUTION, NOT A SEPARATE STEP. A member whose decorated name fails candidate resolution abstains with `no-candidate-match` (367 of 2,350 considered cards in #565's replay) even though a byte-identical sibling resolved. Its tally IS the group's tally, and `resolve_and_persist_illustration` writes the outcome to every member, so it receives the resolved artwork with no path aware it abstained. The rejected alternative - writing a copied `CardIllustrationVote` row - would contribute exactly ZERO weight (it pools under the same agent key; pinned by a test), would manufacture a claim no agent made, and would collide with the model's unconditional (card, anonymous_id) constraint. `Card.inferred_illustration_id` (plain UUIDField, not an FK - no `CanonicalIllustration` table, and no reference-data join to go stale) plus `Card.illustration_vote_status` (migration 0096), written for every group member. `ILLUSTRATION_MIN_VOTES`/`ILLUSTRATION_MIN_SHARE` default to the printing values, changing nothing; there is deliberately no illustration machine weight, since a vote's weight is a property of who cast it, never of what is being voted on. REFERENCE DATA (owner ruling 2026-07-29): this module reads neither `CanonicalCard` nor `CanonicalPrintingMetadata` - it tallies uuids off vote rows and stores the winner verbatim, asserted by `TestReferenceDataIndependence`. A stale snapshot can under-supply the uuids agents have to vote for (upstream) and narrow what a resolved uuid maps to (downstream, a live join every consumer performs itself); it cannot change a tally, flip a winner, or move a propagation. TESTS. 50 new tests, and every one was verified RED against a deliberately mutated implementation (24 mutations; each of the 50 fails under at least one). Coverage includes md5 pooling (agreeing siblings collapse; a self-contradicting agent is withheld order-independently; distinct agents still sum; a group of one is a byte-for-byte no-op including query shape), propagation (a voteless sibling inherits; it does NOT inherit across an identical `content_phash`, with a same-phash-plus-matching-md5 positive control proving the negative is not vacuous), and the human-backed gate for this vote type. HARNESS FIX, and it is not incidental. `test_shared_cache.py::TestSharedCacheTable Migration` carries `@pytest.mark.django_db(transaction=True)`, so its migration round-trip REALLY COMMITS - and its `finally` restored only as far as `0092`, leaving every later migration unapplied for the rest of the session. Latent and free until now purely by alphabetical luck (0093/0094 are data-only; 0095 is read only by modules sorting earlier). A migration adding a `Card` column is the first thing to step on it: `test_sources.py` and `test_stage_e_dispatch.py` sort later and use `transactional_db`, and failed with `column "inferred_illustration_id" ... does not exist` - a message pointing at this branch for a defect entirely in that `finally`. Now restores to `graph.leaf_nodes("cardpicker")`, derived rather than hardcoded. VERIFIED: full `cardpicker/tests/` suite green (3,090 passed, 9 skipped) against a 3,040-passed baseline on the same commit of master. black/isort/ruff/mypy and docs-lint clean. Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
WilfordGrimley
added a commit
that referenced
this pull request
Jul 30, 2026
…cs (#643) `CanonicalPrintingMetadata.face_illustrations` (migration 0095) has been built, merged (#565) and applied in production since 2026-07-29, and the importer already populates it — but nothing said so in a wiki-facing doc, and production still reads 0 populated rows purely because `import_scryfall_printing_metadata` has not run since #565 merged. Records in `docs/features/printing-tags.md` (wiki: `Printing-Tags`): - the two illustration grains on the model — scalar `illustration_id` is front-face-only, `face_illustrations` retains every face — and the two load-bearing properties of the list (the `DOUBLE_FACED_LAYOUTS` gate, and `None` rather than dropping an art-less face so indices hold). - the refresh answer issue #638 left open: the importer is a full-set, value-diffing upsert, and `face_illustrations` is in `_METADATA_SYNC_FIELDS`, so an ordinary run populates already-stored rows. No backfill command and no flag — running the importer is the backfill. - expected coverage, so a mostly-empty column is not later read as a bug: 1,594 of 113,224 rows (1.4%) populate on a first run. Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The defect
stage-d-illustration-v1cast 3CardIllustrationVoterows in its entire existence, against 230,753 catalog cards.local_illustration.pyskipped every card whoseImageEvidence.layout_classwas non-blank, commented as "a multi-faced card has a non-empty layout_class from OCR". The premise is false. That column's only writer islocal_fallback.classify_border_colorand it holds a border colour:blackborderlesswhite''silverNon-blank on 99.34% of rows, so the gate discarded 99.28% of every population handed to the calculator — 3,409 of its 3,426 scanned rows logged
multi-faced-v1. Four other consumers read the column correctly; this was the sole misreader.What shipped
CanonicalPrintingMetadata.face_illustrations(JSONField, migration0095) — every face's ownillustration_idpaired with that face's name, incard_facesorder. The scalarillustration_idstill holds the FRONT face for the four consumers that read it. JSON rather thanArrayField/related rows because the name and the id are correlated (the name is what the index keys a face-named scan on) and two parallel arrays can desynchronise; a partial indexcpm_face_illustrations_presentmakes the new version-stamp term an index-only count.Only genuine double-faced cards get a scannable back — gated on the existing
DOUBLE_FACED_LAYOUTSallowlist.split/adventure/flip/aftermath/mutate/prototypealso nest modes undercard_faces, but those share one printed face, so they get[]: "Stomp" never becomes a second scannable side of Bonecrusher Giant.The gate is deleted, not repaired.
IllustrationIndexnow files each face's artwork under that face's own name, and_resolve_illustration_candidateswidens a back-face-named upload to its combined DFC name for candidates while keying the illustration lookup on the back-face name — so a back-face scan resolves to the artwork actually printed on the side scanned. That removes the wrong-vote exposure the gate stood in for.printings_for_illustrationwidens to match face ids too, so a back-face illustration vote still narrows to its printing.The test that let this ship is replaced.
test_skips_multi_faced_cardsbuilt its fixture withlayout_class="split"— a value the field's only writer can never emit — so it passed while asserting behaviour the production predicate inverted. New tests are parametrised over the field's real vocabulary, plus per-face index tests, import-path tests, and a back-face-named upload end to end.Bumped to
stage-d-illustration-v2.multi-faced-v1is not inRESCANNABLE_SKIP_REASONSand_eligible_illustration_cards_querysetexcludes cards with a non-rescannable scan log for its ownanonymous_id, so a repaired v1 would never re-examine the 3,409 cards it wrongly skipped.models.calculator_familystrips-vN, sopurge_stale_machine_votes,agent_dedupe_keyandresolve_vote_weightfollow the bump automatically — asserted directly inTestCalculatorVersionBump.Verification
pytest cardpicker/— 3002 passed, 6 skipped, 0 failures. mypy, ruff, isort, black all green via pre-commit.DOUBLE_FACED_LAYOUTSallowlist dropped;anonymous_idleft at v1; version-stamp face term dropped;printings_for_illustrationleft scalar-only.no-artist-ocr) and produce 1,920 illustration verdicts + 604 printing verdicts, projecting ~10,277 illustration votes and ~3,233 printing votes catalog-wide — both within 10% of the brief's independently-derived ~10,070 / ~3,610. (A first pass at 2,500 cards projected 2,505 printing votes; with only 39 printing verdicts behind it that was binomial noise, which the 12x sample resolved.)Not merged / not deployed
Owner-only. Note the new column needs an
import_scryfall_printing_metadatarun after migrating before any face data exists.