Per-face illustration ids: the column already works — document the data model, the refresh semantics, and the expected coverage - #643
Merged
Conversation
`CanonicalPrintingMetadata.face_illustrations` (migration 0095) has been built, merged (#565) and applied in production since 2026-07-29, and the importer already populates it — but nothing said so in a wiki-facing doc, and production still reads 0 populated rows purely because `import_scryfall_printing_metadata` has not run since #565 merged. Records in `docs/features/printing-tags.md` (wiki: `Printing-Tags`): - the two illustration grains on the model — scalar `illustration_id` is front-face-only, `face_illustrations` retains every face — and the two load-bearing properties of the list (the `DOUBLE_FACED_LAYOUTS` gate, and `None` rather than dropping an art-less face so indices hold). - the refresh answer issue #638 left open: the importer is a full-set, value-diffing upsert, and `face_illustrations` is in `_METADATA_SYNC_FIELDS`, so an ordinary run populates already-stored rows. No backfill command and no flag — running the importer is the backfill. - expected coverage, so a mostly-empty column is not later read as a bug: 1,594 of 113,224 rows (1.4%) populate on a first run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
An investigation into why
CanonicalPrintingMetadata.face_illustrationsreads 0 populated rows in production, and the doc change that follows
from the answer.
The answer is that nothing is broken. The feature is complete and
merged. What was missing was any wiki-facing description of it, plus a
stated expectation for coverage so a mostly-empty column is not later
misread as a bug. No code change was needed and none is made here.
What already existed (all on
master, PR #565, merged 2026-07-29)0095PrintingMetadataRow.face_illustrationsDOUBLE_FACED_LAYOUTS-gatedimport_scryfall_printing_metadataface_illustrations=_METADATA_SYNC_FIELDSlocal_illustration.pyprinting_metadata__face_illustrationsTestFaceIllustrations(17 tests)Production reads 0 for one reason only:
import_scryfall_printing_metadatahas not run since #565 merged.
Is the importer a full refresh or a diff/upsert? (issue #638)
A full-set, value-diffing upsert — and it does populate existing rows.
_sync_printing_metadatare-derives the desired row for every printing inthe bulk file, streams the stored table, and joins on
canonical_card_id:no match → CREATE; any
_METADATA_SYNC_FIELDSmember differs → UPDATE;all equal → SKIP; stored key not in the desired set → DELETE. The diff is
on values, not on a timestamp or a version marker, so a row whose only
difference is
face_illustrations: [] → [...]is an UPDATE.face_illustrationsis in_METADATA_SYNC_FIELDS, therefore a plainre-run backfills all affected pre-existing rows. No backfill command, no
flag, no migration — consistent with "default the default things".
That membership is the entire load-bearing detail.
bulk_createwritesevery column regardless of the list, so had
face_illustrationsbeenomitted from
_METADATA_SYNC_FIELDS, newly-seen printings would havepopulated normally while all 113,224 already-stored rows stayed
[]forever — a bug that would look exactly like the symptom reported here.
Expected coverage on the first run — 1.4%, and that is correct
Counted from the on-disk
default_cards.jsonthe importer alreadydownloads (zero new Scryfall requests — the file was read in place in
the prod container) and intersected against production, 2026-07-30:
DOUBLE_FACED_LAYOUTS, ≥2 faces)CanonicalCardwith a metadata rowillustration_idon every faceillustration_id: None)CanonicalPrintingMetadatarows totalA first import populates 1,594 of 113,224 rows (1.4%). The other
~111,630 are single-faced and correctly stay
[]— the partial indexcpm_face_illustrations_presentis built for exactly that shape. The 60name-only rows are double-faced layouts for which Scryfall publishes no
per-face artwork id; they record
Nonerather than dropping the face, solist indices keep meaning "face position".
Red-then-green
The brief asked for a test shown red before the change and green after.
There is no change, so instead the existing tests were shown to be
non-vacuous by mutating the two load-bearing lines on
masterandconfirming they fail — the check that would have caught a column silently
staying empty:
master)"face_illustrations"from_METADATA_SYNC_FIELDStest_a_face_illustration_change_alone_is_a_detected_diffface_illustrationsproperty returns[](front-face-only)test_the_import_persists_face_illustrations_to_the_databaseNote the first row: only one test catches the sync-fields omission,
because
bulk_createmasks it for new rows. That single test is whatstands between the feature and a permanently-empty column in production.
Working tree restored; both mutations were local and are not in this PR.
Verification
pytest cardpicker/tests/— 3,416 passed, 8 skipped (4m06s)docs_lint.py --strict— clean0095already exists and is applied, so thecontended
0100–0103range is untouched and PR Fail CI on a migration graph that forks WHEN MERGED, not just on the branch #611's leaf guard isunaffected.
production, no Scryfall requests.
The one operational follow-up
Populating the column requires an ordinary
import_scryfall_printing_metadatarun. That is a write and was deliberately not performed here.
🤖 Generated with Claude Code
https://claude.ai/code/session_013NhYmT1PxCcyemA16dFDxN