Skip to content

E-2: backend zero-hit printing-filter degradation - #62

Merged
WilfordGrimley merged 3 commits into
masterfrom
claude/e2-zero-hit-printing-degradation
Jul 18, 2026
Merged

E-2: backend zero-hit printing-filter degradation#62
WilfordGrimley merged 3 commits into
masterfrom
claude/e2-zero-hit-printing-degradation

Conversation

@WilfordGrimley

@WilfordGrimley WilfordGrimley commented Jul 18, 2026

Copy link
Copy Markdown

Description

Fixes live behavior today: a printing-specific decklist paste (e.g. Moxfield's (SET) NUM suffix, or E-1's <set>/<collectorNumber> XML import) silently finds nothing when nobody's uploaded an image for that exact printing yet — even though the card exists under other printings. search_functions.get_search applies expansion_code/collector_number as a hard Elasticsearch .filter("term", ...); zero hits under that filter meant zero hits, full stop.

retrieve_card_identifiers now retries once, without the printing filter, whenever the filtered search returns zero hits and a printing filter was actually supplied — and reports a degraded flag. Exact-match behavior when hits DO exist under the filter is completely unchanged — this is a zero-hit fallback only, never a boost/re-rank weakening (_resolved_printing_match_tier untouched).

EditorSearchResponse gains degradedQueries: string[] (hash keys whose search degraded), threaded through the schemas/ JSON-Schema → quicktype → Python/TS pipeline. Backend-only in this PR — frontend badge consumption ("image unavailable, matched printing instead" style UI) is natural follow-up work, not wired here.

Testing note (please read before reviewing the test diff)

This sandbox has no working Docker daemon (confirmed: dockerd fails to start with a permission error) — the existing pytest suite's testcontainers-based Postgres/Elasticsearch fixtures (conftest.py) can't run here, despite a bare Postgres 16 cluster being available on the host (started it, confirmed it doesn't help — the fixtures need their own ephemeral containers regardless). This matches this repo's own documented cloud-session constraint.

What I verified instead:

  • A standalone script that imports the real search_functions module directly (bypassing the Docker-dependent conftest.py entirely) and exercises retrieve_card_identifiers's retry logic against a mocked _retrieve_card_identifiers_once — 4 scenarios (hits found/no retry, zero hits/retries+degrades, no filter supplied/no spurious retry, expansion_code-only still retries), all passing.
  • mypy/black/isort/ruff run locally at the exact versions pinned in .pre-commit-config.yaml (isort 5.12.0, black 22.8.0, mypy 1.7.0, ruff 0.0.257) — 0 new errors on every touched file; confirmed the 4 pre-existing mypy errors that do show up are all in files this PR never touches (models.py, tag_consensus.py, mtg.py).
  • py_compile on every touched Python file.
  • npx tsc --noEmit on the frontend (the regenerated schema_types.ts compiles clean; nothing currently reads the full EditorSearchResponse shape, only .results, so this is a safe additive change on the frontend side today).

New pytest coverage is included (test_search_functions.py: 4 new tests on the retry/degrade decision in isolation; test_views.py: 2 existing snapshot tests updated for the correct new behavior + 1 new multi-query test asserting degradedQueries tracks per-query, not globally) along with hand-patched snapshot fixtures (test_views.ambr — 25 purely-additive degradedQueries: [] entries, plus the 2 existing "no results" tests corrected to reflect that they now find the card via the degraded fallback). I wrote these to match the exact syrupy snapshot format already in the file, cross-checked against ~25 existing examples, but they have not been executed.

Merge-time checklist

  • Run the full test_search_functions.py/test_views.py pytest suite in a real environment with Docker (or CI) to confirm the hand-authored snapshot entries are byte-correct and every new test actually passes. DONE (2026-07-18, real Docker/testcontainers on the host, not this sandbox): 845 passed, 4 failed (2 pre-existing moxfield-flake, 2 pre-existing missing-client_secrets.json - both confirmed via a bare-master control run, unrelated to this PR). Found and fixed a REAL regression during this gate: the new test_only_the_actually_degraded_query_is_flagged_among_several test shifted factory.Sequence counters for 6 downstream, unrelated snapshot tests (TestGetSampleCards/TestNewCardsFirstPages/TestNewCardsPage/TestPostExploreSearchResults) - fixed via a name-gated module-level autouse fixture that insulates just the new test (d7e4653c; documented as a troubleshooting.md variant, 01bc1fdf). 130/130 snapshots pass after the fix, zero .ambr regeneration needed. Schema regeneration also verified consistent: npx tsx quicktype.ts output diffed against committed schema_types.py/schema_types.ts - identical modulo pre-commit formatting (black/prettier/isort import order), including the new degradedQueries field.
  • Re-scoped to POST-DEPLOY, DONE (2026-07-18, after this PR merged + backend redeploy - b884a779 baked as GIT_SHA, confirmed via /2/info/ health check and showmigrations before/after): curled the live 3/editorSearch/ endpoint (post_editor_search, this PR's own function) with a printing-specific query guaranteed to zero-hit under the filter - {"query": "Brainstorm", "cardType": "CARD", "expansionCode": "ZZZ"} ("ZZZ" is not a real expansion code, same technique this PR's own test_only_the_actually_degraded_query_is_flagged_among_several uses). Response: {"degradedQueries": ["key1"], "results": {"key1": ["10H3ZbWslBZlNnYFUpZfNU5pxLBx-lI6v", "101iJliFXUOgp-GXXf4Du7nE3t-3HtxyQ", "1R4kpXBng-uRY5JsUeFkhBTbEnwmdDAiM", "1R6v5gjF4Bu5rktuomvkp2OXQd0IcQwuR"]}} - same 4 results as the unfiltered name-only search, key1 correctly flagged in degradedQueries. Confirmed against the real production Elasticsearch index, not fixture data.

Generated by Claude Code

search_functions.py: when a printing-specific search (expansion_code
and/or collector_number supplied) finds zero hits under Elasticsearch's
hard filter, retrieve_card_identifiers now retries once without the
filter and returns a `degraded` flag alongside the identifiers -
fixing live behavior today, where a printing-specific decklist paste
(e.g. Moxfield's "(SET) NUM" suffix) silently finds nothing for a card
that exists under other printings, just not this exact one yet.

Exact-match behavior when hits DO exist under the filter is completely
unchanged - this is a zero-hit fallback only, verified via the
existing _resolved_printing_match_tier re-rank logic being untouched
and covered by new tests asserting the retry never fires when the
filtered search already found something.

EditorSearchResponse gains a new required `degradedQueries: string[]`
field (hash keys whose search degraded) via the schemas/ pipeline
(JSON Schema -> quicktype -> Python/TS). Backend-only in this PR -
frontend badge consumption is natural follow-up work, not wired here.

Testing note: this sandbox has no working Docker daemon (confirmed:
dockerd fails to start), so the existing pytest suite's testcontainers-
based Postgres/Elasticsearch fixtures can't run here despite a bare
Postgres 16 cluster being available on the host. Verified instead via:
a standalone script that imports the real module and exercises the
retry logic against mocked `_retrieve_card_identifiers_once` calls (4
scenarios, all passing); mypy/black/isort/ruff run locally at the
exact versions pinned in .pre-commit-config.yaml (0 new errors -
confirmed the 4 pre-existing mypy errors are all in untouched files);
py_compile on every touched file. New pytest coverage (test_search_
functions.py, test_views.py) and hand-patched snapshot fixtures
(test_views.ambr - 25 additive `degradedQueries: []` entries plus 2
behavior-corrected entries for the two existing filter-with-no-match
tests, which now correctly find the card via the degraded fallback)
are included and were written to the exact syrupy snapshot format
already in the file, but have NOT been executed - live pytest
confirmation is a merge-time requirement, not a nice-to-have, given
none of this could run end-to-end locally.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014dB7mGPQyoML2Ri5ZrprdK
@WilfordGrimley
WilfordGrimley marked this pull request as ready for review July 18, 2026 16:42
@WilfordGrimley
WilfordGrimley merged commit b884a77 into master Jul 18, 2026
5 of 6 checks passed
@WilfordGrimley
WilfordGrimley deleted the claude/e2-zero-hit-printing-degradation branch July 18, 2026 16:42
WilfordGrimley added a commit that referenced this pull request Jul 18, 2026
…staleness mini-pass (#79)

* publish_wiki.py: fix link rewriting for parent-relative paths and slug casing

GitHub wiki's native [[...]] auto-linking reinterprets our own docs/
[[file.md]] convention: [[../troubleshooting.md]] became a dead literal
slug, and [[printing-tags.md]] linked the raw filename casing instead of
the published page name "Printing-Tags" (both visible live on
Catalog-Completion-Plan). Every internal link is now resolved against
its source file's real repo path and mapped through
wiki-publish-map.json: a published target becomes a same-wiki link using
its real page name, an unpublished-but-real target becomes an absolute
GitHub blob URL, and a target resolving to neither is a hard publish
error. docs_lint.py can't catch this class of bug since it only checks
links within docs/ itself, not the wiki-transform's own reinterpretation
downstream - the script now self-checks at publish time instead.

Also adds two docs/ pages that were listed in docs/README.md's index
but missing from the mapping entirely (found via the new validation):
documentation-process.md and upstream-wiki-drift.md.

* Docs staleness pass: Part 3 write-pass status + proposal B/C/E-1/E-2 markers

catalog-completion-plan.md: Part 3 heading and Status section updated to
reflect the completed write pass (run_id 20260718T145157-a12b1387, 13,275
votes, all hard bounds passed, 0/7,124 zero-resolution violations at full
population), pointing to docs/reports/2026-07-18-part3-write-pass-complete.md.
Filled the dangling "see the follow-up entry below" reference and replaced
the now-stale "HOLD #P3 stands" language. Part 4's heading now notes it's
confirmed unstarted with HOLD #B prep queued.

proposal-b-bleed-normalization.md: top summary no longer lists the
prior-resolution batch fetch as remaining work - it shipped as PR-1 (#72).

proposal-g-user-accounts-saved-decks.md: noted the build-order queue
(E-1 #61, E-2 #62, Level-2 grid fix #63, audit pass #64, GIS error UX #65,
Proposal B #66/#72, Proposal C part (a) #67) has fully cleared.

docs/README.md: added proposal-b and proposal-c to the "Plans & proposals"
table - both have dedicated docs but were missing from the index entirely,
despite the section's own stated policy that any proposal with a dedicated
doc gets a row.

printing-tags.md and vote-system.md checked per the mini-pass's minimum
list; no changes needed - printing-tags.md already defers Stage 8+ status
to catalog-completion-plan.md, and vote-system.md has no AI-terminology or
merged-PR staleness.

* Report relay: wiki review findings (publish-script fix, Part 3 status, staleness mini-pass)

* CI: black-format publish_wiki.py, prettier-format cache-transition-resilience.md

black reformatting for publish_wiki.py (never run locally - only
py_compile was checked, missing the repo's black rev 22.8.0 pinned in
.pre-commit-config.yaml). cache-transition-resilience.md's prettier
drift predates this PR (introduced by #75) and is unrelated to its
content - swept here since the "Formatting and static type checking"
check runs pre-commit against all files, not just the diff, and this
PR's own CI needs to go green.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants