Troubleshooting

Troubleshooting

Symptom-first index. Grep this file for your error text or observed behavior before re-deriving a fix from scratch. Each entry: symptom → cause → fix → refs.

Why symptom-first, not cause-first: several of these entries were already documented (in docs/lessons.md or a feature doc) by cause the first time they happened, and still recurred — sometimes 2-3 more times — because a worker hitting the symptom didn't know which cause-shaped heading to look under. Indexing by what you actually see when you hit the problem is the fix for that; grep the exact error text, not a guess at its root cause.

CI mypy fails on a package that "isn't even used" / a stub-type error that never reproduces locally

Symptom: the "Formatting and static type checking" GitHub Actions workflow fails on a package that seems unrelated to your change, or a type error you've never seen locally — and a fresh local pre-commit run mypy (or full local test run) stays green.

Cause: mypy's pre-commit hook has its own isolated dependency resolution (.pre-commit-config.yaml's additional_dependencies) and mypy_django_plugin genuinely imports (not just statically analyzes) everything reachable from cardpicker/models.py to introspect Django models. Any transitive import reachable from that chain but not listed in the hook's additional_dependencies is a hard ModuleNotFoundError in CI's isolated hook env — while your local venv (installed from the full requirements.txt for pytest) silently has it, masking the crash completely. Trust CI history over a matching local run for anything touching the models.py import chain.

Fix: add the missing transitive dependency to .pre-commit-config.yaml's mypy additional_dependencies (and usually pin it in requirements.txt too). Verify via gh run list/gh run view --log — not just a local re-run — that CI is actually clean afterward.

Recurred 3 times: Pillow via cardpicker/models.py (fixed by 9837a4cd, follow-on real type errors fixed by 40e04d54; documented Lessons), numpy via imagehash (24e7d293), the hash-at-ingest import chain (update_database → local_phash → imagehash/pytesseract, journal/2026-07-16-hash-at-ingest.md).

5-6 unrelated test snapshots break after adding one new test file (RETIRED 2026-07-23)

This whole class of bug is now structurally impossible — see "Root fix" below. Kept as history because the root cause (factory.Sequence counters being process-global) is a generically useful fact about factory_boy, and because the old symptom text is still what you'd grep for if you hit something snapshot-related in this suite.

Old symptom: a brand-new test file (using an existing shared factory) was added, and several other, seemingly-unrelated tests started failing — often test_views.py::TestGetTags::* or similar snapshot-style assertions with a hardcoded value like "Artist 0".

Cause: factory.Sequence counters in cardpicker/tests/factories.py are process-global for the whole pytest run. test_views.py is the only module in the suite whose assertions embed a sequence-derived value (via __snapshots__/test_views.ambr, reached through brainstorm_canonical_card's default CanonicalCardFactory/ CanonicalArtistFactory SubFactory chain) — so that value implicitly depended on total call count up to that point in collection order, and any other file using the same shared factories could shift it.

Old fix (retired): an autouse fixture local to every new test file that captured each shared factory's next_sequence() before the test body ran and called reset_sequence(n, force=True) both immediately and again in teardown, keeping that file's own usage invisible to the rest of the suite. This recurred 3 times after being documented (Lessons) because each new file had to independently rediscover which factories count as "shared" (deductive-backfill work, then again in test_purge_machine_votes.py), and needed a special-cased, request.node.name-gated variant (test_views.py's old _preserve_shared_factory_sequences_for_insulated_tests) for the case of a single new test inside an existing file, since a same-scope populated_database-consuming test shifted every later test in the same file.

Root fix: the burden was on the wrong side. Instead of every module that merely uses the shared factories protecting the one module that asserts on their exact values, test_views.py now pins those factories to a fixed baseline (Factory.reset_sequence(0, force=True)) before every one of its own tests (_pin_shared_factory_sequences, module-level autouse). Its snapshots are now self-determined regardless of suite composition, collection order, or how many tests ran before it — no other file in cardpicker/tests/ needs to know test_views.py exists, and the old capture/restore fixture + _SHARED_FACTORIES list was deleted from all 31 other files that carried it. This also retired the --snapshot-update single-file-vs-full-suite divergence variant that used to apply here (updating test_views.ambr in isolation used to bake in wrong values because a full-suite run consumed sequence numbers a single-file run didn't) — since the pin always resets to the same baseline regardless of what ran before, pytest cardpicker/tests/test_views.py --snapshot-update and a full-suite --snapshot-update now produce identical output, both scopes work.

Seeding rows via a data migration breaks tests that assert a table is empty/complete

Symptom: a new data migration seeds rows into a table, and several unrelated tests that assert the table's complete contents in a fresh DB start failing.

Cause: a migration runs unconditionally at DB-setup time, including the test database — any migration-seeded row becomes permanent baseline state for every test in the suite, not just ones that care about it.

Fix: seed via a manual, idempotent management command (get_or_create + a thin wrapper), never migrations.RunPython, regardless of how literally a task spec says "data migration." Follow the existing seed_default_tags/seed_no_match_reason_tags/ seed_attribute_tags pattern.

Recurred: hit and documented once for Tag seeding (Lessons), and independently again the same day (journal/2026-07-14-tag-taxonomy-followup.md) before the lesson had even been written down.

nginx 502s everything after a django container restart

Symptom: every API request 502s after docker compose up -d django worker (or anything that recreates the django container) — connect() failed (111: Connection refused) ... upstream: "http://<stale-ip>:8000/..." in nginx's logs.

Cause: nginx's upstream django-api { server django:8000; } resolves the django service name to a Docker-internal IP once at nginx's own startup/reload, not per-request. Recreating the django container assigns it a new internal IP; nginx keeps proxying to the old one.

Fix: sudo docker compose -f docker-compose.prod.yml restart nginx after any command that recreates the django container. See Infrastructure's Docker/backend deploy section.

TesseractNotFoundError / OCR tests fail without the real tesseract binary

Symptom: TesseractNotFoundError inside the mpcautofill_django container, or OCR-dependent tests fail in CI specifically (not locally).

Cause: tesseract-ocr wasn't originally in the Docker image's build stage, and CI test runners also lack the real binary. Important distinction: the mpcautofill_django Docker image DOES have the real binary baked in now, but the "Backend tests" GitHub Actions job (.github/actions/test-backend/action.yml) never uses that image — it's a bare ubuntu-latest runner with pip install -r requirements.txt and nothing else, so it never has tesseract and never will unless that action is changed. A local check against the Docker image (which has the binary) will not catch a missing mock; only CI, or a local repro with pytesseract.pytesseract.tesseract_cmd pointed at a bogus path, will.

Fix: tests mock tesseract directly rather than requiring the real binary in CI (ddb6dce9, "Fix CI: mock tesseract in tests") — any test whose code path reaches local_ocr.run_tesseract (directly, or transitively via local_fallback.detect_illus_anchor's unconditional call whenever fetch_card_image returns a non-None image) must monkeypatch.setattr(local_ocr_module, "run_tesseract", lambda image: "<expected text>"). This has already recurred once past the original fix: 10 tests added in test_local_identify_printing_tags.py across 2026-07-15 to -17 (e6b09d14, 3b2b5b7d, c7010bd8) called run_pilot with a real fetched image and skipped the mock, passing locally (host venv/Docker image both have the binary) but failing CI outright — fixed 2026-07-19. See Printing-Tags's build history (git log e4eb6cb3 -- docs/features/printing-tags.md) for the full timeline if you need it.

prettier rewrites already-correct markdown into broken text

Symptom: running the prettier pre-commit hook on a markdown file changes text that was already correct — e.g. *italic* becomes a broken \_italic*, or node_modules gets mangled — especially on a second pass over a file prettier already formatted once.

Cause: a real non-idempotency bug in prettier@2.7.1 on specific text patterns (backtick-adjacent underscores/asterisks in particular).

Fix: reword the two known trip points (backtick-wrap the term, or use **bold** instead of _italic_/*italic*) rather than fight the formatter. Verify by re-running the hook and confirming zero further diff. See Lessons for the specific patterns found so far.

A Playwright test hangs on a stuck loading spinner / blank src="" image, and the same test passes in an isolated worktree

Symptom: a Playwright test times out with <div class="spinner-border"> ... subtree intercepts pointer events blocking a click on a card image, and page.getByAltText(...) resolves to an <img src=""> that never finishes loading — reproducible several times in a row against the session's long-running next dev server on port 3000, for a test file that has nothing to do with your actual diff.

Cause: a next dev process that has been alive for a long time (many hours, dozens of Playwright runs, repeated file edits triggering fast refresh) accumulates state that isn't reproducible in a freshly-started server — confirmed by swapping frontend/src between a branch and master on the same live server (both failed identically) versus running the identical test in a brand-new git worktree with its own freshly-started next dev on a different port (passed 30/30, twice each, both master and the branch). The failure tracks the server process's age/history, not the source code. Likely candidates: stale webpack HMR module chunks, or accumulated fetch/image-cache pollution from earlier test runs' mocked network state leaking across the dev server's lifetime — not confirmed further since the worktree test was decisive enough to stop investigating.

Fix: before concluding a test failure is a real regression, reproduce it in a fresh, isolated git worktree with its own newly-started dev server on a different port (the same technique used for before/after screenshots — see Lessons's worktree-dev-server-collision entry), not just by editing files in place on the session's existing long-running server. If the isolated run passes, restart the main session's dev server (kill the old next dev/next-server PIDs, start fresh) rather than trusting further results from the stale one. Don't spend time root-causing the exact HMR/cache mechanism unless it recurs after this fix.

Addendum (2026-07-22, /whatsthat animation-sync fix): this exact symptom — revealed (backing question-feed's loading spinner) stuck false forever, <img src=""> never settling — recurred on a genuinely fresh, isolated dev server (a brand-new worktree, freshly killed and restarted, confirmed via ps/readlink -f /proc/<pid>/cwd that no other session's server was reused), so it was not always the stale-server artifact described above. The real cause that time: a useEffect keyed on item?.card.identifier doing settle/gate logic whose reset lived unconditionally in the fetch handler (setRevealed(false) etc. on every resolution, not just ones landing on a genuinely new identifier). Two consecutive feed items can legitimately share an identifier (the existing fetch-handler comment already documents this for chipStates), and dev-mode React Strict Mode's double effect invocation makes a duplicate resolution routine even outside that case — when it happens, the reset fires again but the identifier-keyed catch- up effect has no dependency change to re-trigger it on, permanently stranding the reset state. Fix: don't key a catch-up/settle effect on a value that can legitimately repeat between consecutive items — key it on a counter bumped unconditionally in the same reset block instead (imageGeneration in QuestionFeed.tsx), so the effect re-runs every time the reset does, with no dependency on whether the identifier text itself changed. Distinguishing the two causes: the stale-server version reproduces identically regardless of source code (swapping branches on the same long-running server fails the same way); this version reproduces intermittently even on a fresh server and stops reproducing (verified via a 15-iteration --workers=1 loop with zero source edits mid-run) once the generation-counter fix lands — a single clean pass proves nothing for an intermittent race like this, only a multi-iteration loop does.

npx prettier --write reformats far more of a frontend file than you touched (trailing commas, wrapped ternaries appearing everywhere)

Symptom: running npx prettier --write on a file you made one small, targeted edit to produces a diff spanning dozens of unrelated pre-existing lines — trailing commas added to multi-line function calls, ternary expressions gaining parentheses, multi-value CSS properties reformatted onto separate lines — none of which you changed.

Cause: frontend/'s actual formatting contract is prettier 2.7.1, pinned via .pre-commit-config.yaml's pre-commit/mirrors-prettier hook (rev: "v2.7.1") — not via frontend/package.json, which doesn't list prettier as a direct dependency at all. frontend/node_modules/.bin/prettier resolves to whatever transitive version package-lock.json happens to pin (confirmed 3.7.4 on master as of 2026-07-17) — a different major version with different defaults (trailingComma: "all" vs "es5", plus 3.x's wrapped-ternary and multi-value-transition formatting changes). Since none of frontend/'s actual source files are formatted to 3.x's rules, running the wrong binary against any file reformats every pre-existing line it touches, not just your edit.

Fix: always run npx --yes prettier@2.7.1 --write <files> explicitly in a sandbox without pre-commit installed — never bare npx prettier or node_modules/.bin/prettier. Verify with npx prettier@2.7.1 --version prints 2.7.1 before trusting a diff as edit-scoped. If a diff already shows this kind of unrelated mass reformatting, it's a signal to check the prettier version immediately, not to assume the file was simply out-of-date.

Worker-thread DB queries silently miss data / "too many clients already"

Symptom: either (a) a test using a ThreadPoolExecutor for a DB query passes locally but the worker thread sees no rows under pytest-django's default db fixture, or (b) in production, psycopg2.OperationalError: FATAL: sorry, too many clients already crashes a long-running command.

Cause: Django DB connections are thread-local. (a) A worker thread can't see an uncommitted test transaction under the default db fixture. (b) A ThreadPoolExecutor constructed inside a per-chunk loop (instead of once for the whole run) leaks one Postgres connection per chunk, since nothing closes a thread-local connection when its thread is torn down — this exhausts max_connections over a long run.

Fix: (a) use the transactional_db fixture instead of db for any test that touches the DB from a worker thread — same fix as an existing test_sources.py precedent (journal/2026-07-15-local-printing-id-pilot.md). (b) hoist ThreadPoolExecutor construction to wrap the entire loop, not per-chunk — see Printing-Tags's build history for the incident this was found in.

Entrypoint + migrate composition traps (boot-time rescan, deploy/migrate fusion)

Two distinct symptoms, same root subsystem: docker/django/entrypoint.sh always runs migrate before anything else on django/worker start.

Symptom 1 — API unreachable for 10+ minutes after a deploy: a purely schema-only migration (e.g. one nullable column) triggers a full catalog rescan across every source before gunicorn binds; if a per-source IntegrityError crashes the container mid-rescan, it stays down (no restart policy) with docker compose logs looking identical to a slow-but-alive container on casual inspection.

Cause 1: entrypoint used to gate import_sources/update_database/update_dfcs behind migrate --check ("did any migration apply") — the wrong proxy for "does catalog content need rescanning."

Fix 1: entrypoint now only runs migrate + import_sources before binding; content sync is scheduled (daily/weekly django-q jobs) plus a fresh-bootstrap-only guard (eaece1fd, #18). Hardened a week later with restart: unless-stopped on every service (8b1ec5e5) plus a systemd unit + verified reboot test (ac6bb7e3). See Infrastructure's "Startup vs. scheduled catalog sync" and "Boot-time recovery" sections for current behavior.

Symptom 2 — a live long-running job crashes mid-query with column ... does not exist right after an unrelated deploy: recreating the persistent django/worker containers to ship new code applied a column-rename migration (Card.image_hash → content_phash, PR #27) as an unintended side effect, while a separate one-off container (docker compose run --rm worker ..., a live full-catalog pilot job) was still running the old image against the same database. The rename executed mid-query under the live job, which crashed on its next read.

Cause 2: entrypoint's fix for Symptom 1 (above) made migrate run unconditionally on every django/worker start, by design (PR #18 — so a container that failed to migrate self-heals on its next boot). That means docker compose up -d django worker is a fused deploy+migrate step with no way to do one without the other — a "build → deploy → migrate" plan that assumes three separable steps may already be two.

Fix 2: never recreate the persistent django/worker containers while any other container on an older image is actively running against the same database, unless every pending migration is strictly additive (nullable column, new table — anything old-code ORM simply ignores). For a non-additive migration (rename, type change, NOT NULL backfill): stop the long-running job first and restart it after the deploy, or apply the migration manually from an old-code container before recreating anything. Check entrypoint.sh (or equivalent) before assuming deploy and migrate are actually separable. (8c957aa5, 2026-07-16.) Even when Fix 2's own condition holds (strictly additive), starting django and worker together (docker compose up -d django worker) still races their two entrypoint migrate steps against each other — observed once, 2026-07-17, as one container crashing on column ... already exists before self-healing via restart: unless-stopped; harmless for an additive migration, but worth knowing the crash-then-recover blip is expected, not a new problem.

Related gotcha, same incident: GIT_SHA=$(git rev-parse --short HEAD) sudo docker compose build ... bakes unknown instead of the real SHA — sudo doesn't preserve environment variables set before it on the same command line. Use sudo env GIT_SHA=$SHA docker compose build ... or sudo -E instead. Cosmetic only (git-SHA baking is best-effort visibility, never the staleness guard itself), but silently wrong if unfixed.

Frontend ships an API shape change before the backend redeploys ("undefined cards"-class bugs)

Symptom: a page renders a literal undefined where a number/value should be (e.g. "undefined cards"), immediately after merging a PR that changes an API response shape — with no code regression in the PR's own tests, which all pass.

Cause: the frontend (GitHub Pages) and backend (persistent django/worker containers) deploy on separate pipelines. Pages auto-ships on merge/push to master; the backend only picks up a merged change once someone explicitly redeploys the persistent containers (docker compose up -d django worker). A frontend-only merge that changes an API response's shape is live on Pages within minutes, but the backend can lag by hours or longer if nothing triggers a redeploy — "stale but schema-compatible" is false the moment a shape change merges; that assumption only holds for the deploy-skew window before the shape actually changed, not after.

Fix: (a) merging an API-shape-changing PR isn't "done" — it isn't complete until the persistent containers are explicitly redeployed to match; track that as a first-class follow-up, not an implicit side effect of the merge. (b) independent of (a), any frontend consumer of a versioned/typed API response should defensively handle the previous shape at runtime — TypeScript's compile-time cast can't catch a live shape mismatch against whatever the backend is actually serving right now.

Incident: 2026-07-16/17, questionFeed's remainingEstimate field changed from a plain number to a QuestionFeedCounts object (#29). Pages shipped the new frontend immediately; the persistent backend containers were still serving the old plain-number shape. QuestionFeed.tsx's headline read counts.total on what was actually a raw number at runtime, rendering the literal string "undefined cards" in production. Fixed on the frontend side with a runtime shape guard (normalizeQuestionFeedCounts(), #34) that degrades gracefully to the old copy when it detects the legacy shape — worth keeping permanently, since every future backend deploy has the same skew window, not just this one incident.

A quicktype-generated frontend type is "missing" a field

Symptom: a PrintingCandidate/Tag/etc. TypeScript type (generated by quicktype) doesn't have a field that "should obviously already be in the payload," or a schema field you hand-added to the .ts file directly disappears on the next build.

Cause: schema_types.py/schema_types.ts both say "Generated by quicktype. Do not manually modify this file." — the source of truth is the JSON Schema under schemas/schemas/.

Fix: edit the JSON Schema source file, then cd schemas && npm run build. Run black/isort/prettier on the output afterward — raw quicktype output isn't formatted, so an unformatted diff is mostly noise, not the real change.

Recurred 3+ times: see Printing-Tags's build history and journal/2026-07-14-tag-taxonomy-followup.md.

Bulk fetch through image-cdn runs faster than the configured rate limit, zero 429s

Symptom: a bulk fetch job against the image CDN (e.g. the content_phash backfill) sustains a throughput well above the configured ceiling (observed ~10.5/s against a configured 3/s), with zero 429/rate-limit-rejection lines anywhere in the job's own log, for an extended period (50+ minutes observed) — not a brief burst.

Cause: the Worker's IMAGE_FULL_TIER_RATE_LIMITER binding (image-cdn/wrangler.toml, namespace_id = "1002", simple = { limit = 30, period = 10 } = 3 req/sec — config confirmed directly from the repo file, the Cloudflare dashboard exposes the binding's existence/namespace but not its configured limit/period) is not enforcing its configured ceiling at this volume. Two specific application-level bug hypotheses were checked and both ruled out by direct code read, not left as open guesses:

1. Routing bypass — confirmed as a real, separate cause (see above): get_worker_image_url always builds the /images/google_drive/full/... URL regardless of dpi, and the Worker's "full" case has no cache short-circuit, so every request does unconditionally reach fetchWithRateLimit.
2. Per-key scoping — checked 2026-07-17, not the cause: every caller of fetchWithRateLimit against this specific limiter was enumerated (grep -rn "fetchWithRateLimit\|\.limit(" image-cdn/src/ — exactly one call site exists, image-cdn/src/handler/image.ts:45) and its key argument is the literal string "global-image-full-tier-rate-limit" — a fixed, shared constant, not a per-URL/per-card value. Inside fetchWithRateLimit itself (image-cdn/src/utils.ts:17, limiter.limit({ key })), the same key parameter is reused across every retry attempt in the loop too. So every full-tier request, across every retry, genuinely shares one counter — a fresh-counter-per-image bug would explain the symptom, but the code does not have that bug.

With both application-level hypotheses ruled out and the dashboard confirming the binding itself exists at the right namespace with (per the repo config) the right limit/period, the remaining explanation is Cloudflare's Rate Limiting binding not enforcing atomically/globally at this request volume — a documented characteristic of the product at scale, but not confirmable further in this environment: wrangler here requires Node 22+ (box has 20.20.2) and full CLOUDFLARE_API_TOKEN/dashboard request-analytics access is unavailable.

Fix (working control, 2026-07-17): client-side pacing at the fetch call itself — cardpicker/local_phash.py's _RateLimiter (a strict minimum-interval pacer, not a token bucket, shared across every worker thread) plus DEFAULT_BACKFILL_RATE_LIMIT_PER_SEC, wired through run_content_phash_backfill's rate_limit_per_sec param and the local_backfill_content_phash command's --rate-limit-per-sec flag (default matches the Worker's own configured-but-non-enforcing 3/sec). Any other bulk caller of the image CDN's full tier should assume the same and add its own client-side pacing — don't rely on the Worker binding alone at bulk volume. No Worker-side code fix is queued for this specific gap (the key-scoping fix that would normally follow this kind of diagnosis doesn't apply here — the key was already correct); if Cloudflare dashboard/API access becomes available later, revisit whether the binding itself needs a support ticket or config change.

Refs: docs/features/image-cdn.md, docs/features/catalog-completion-plan.md's Part 2 section.

A cloud-sandbox Playwright run fails ~24-29 specs that all pass locally with real network

Symptom: a batch of otherwise-unrelated Playwright specs (AddCardToFavorites, ArtistVotePicker, PrintingTagPicker, TagVotePicker, ReportCard, Toasts, CardDetailedViewModal.visual, intermittently New.visual) fails together, every time, in a given cloud/agentic dev sandbox — never a partial subset, never a different failure signature — while GitHub Actions CI passes the same specs cleanly on the same code.

Cause: the sandbox environment these specs were authored in has no real network egress to the image-CDN domains (cdn.proxyprints.ca/img.proxyprints.ca) the app fetches card images from — every card image fails to load regardless of the diff under test, producing a consistent, diff-independent failure signature. One representative failure traced directly to a getByAltText call timing out waiting on a real image load (see PR #35's body for the full investigation). Confirmed as the correct diagnosis, not just a plausible guess: re-ran this exact bucket (26 specs including all of the above) on 2026-07-17 from this box, with a real dev server pointed at real NEXT_PUBLIC_IMAGE_WORKER_URL/NEXT_PUBLIC_IMAGE_BUCKET_URL values and genuine outbound network access — all 26 passed. GitHub Actions CI runners have real egress too, which is why this bucket has never shown up there.

Fix: there is nothing to fix in application code — this is an environment property, not a bug. Don't chase these failures as regressions when they show up in a sandbox; don't add sandbox-specific skips or mocks to work around it either, since that would silently weaken the tests everywhere. To get a genuine real-image signal from this kind of environment, either run against a real dev server with real CDN env vars and real egress (this box, or any environment with outbound network access), or trust GitHub Actions CI's own run.

Refs: PR #35's body (root-cause investigation), PRs #36/#37/#41 (reference back to #35 rather than re-investigating).

A worktree merge silently loses its merge state / a push produces a single-parent commit with the right content but the wrong git history

Symptom: after resolving a real merge conflict in a git worktree (a directory whose own .git is a file pointing at the real gitdir, not a directory) and pushing the result, GitHub still reports the PR as CONFLICTING/DIRTY even though the pushed commit's file content is verifiably correct (a direct git diff against the target branch shows exactly the expected changes, nothing missing).

Cause: writing directly to .git/MERGE_HEAD (e.g. via git rev-parse origin/master > .git/MERGE_HEAD, intending to manually restore merge state so the next git commit produces a real 2-parent merge commit) silently fails in a worktree, because .git there is a file containing a gitdir: <path> pointer, not a directory — the shell redirect (>) can't write into it, and by default that failure doesn't stop the rest of the command chain. The following git commit still succeeds, but as an ordinary single-parent commit (parent = the branch's own prior tip), not a merge commit. The tree/content is correct — it came from a real 3-way merge whose resolution was preserved via a stash — but the commit graph no longer shows the target branch as an ancestor, so GitHub's mergeability check performs its own fresh 3-way merge attempt against the current base and re-encounters the original conflict.

Fix: check git log -1 --format="%P" <commit> after any commit you expect to be a real merge — it should list two parent SHAs. If it lists only one where two were expected, the tree is still salvageable (assuming a real merge actually happened first, e.g. via git merge, and you have the resulting SHA or a stash of it): rebuild a correct 2-parent commit directly with git commit-tree <tree-sha> -p <parent1> -p <parent2> -m "...", reusing the already-verified tree rather than re-doing the merge. Verify with the same %P check before pushing. If the flawed single-parent commit was already pushed, this requires a force-push to replace it — treat that with the same care as any other force-push (fresh, explicit confirmation first), even though the tree content itself is unchanged.

Refs: none yet — first occurrence, 2026-07-17, during the frontend-package PR #41 conflict resolution.

User reports "Vote failed" / a generic tag-submission error — check nginx access logs for 429s first

Symptom: a real user reports a vote/tag submission failing with a generic frontend toast ("Vote failed" / "Something went wrong submitting your tag - please try again.") and docker logs mpcautofill_django shows nothing around the reported time.

Cause: the empty django logs are a red herring, not evidence of a missing traceback — check docker logs mpcautofill_nginx for the actual HTTP status first. In the one confirmed case so far (2026-07-17, Changeling Outcast border/frame chips), the real cause was a 429 from cardpicker.views.post_submit_tag_vote's rate limit (a real user voting quickly enough to trip it), not a 500 — the backend behaved exactly as designed (a clean, well-shaped JSON error, checked before any request-body parsing or vote-casting) and there was never an exception to log anywhere. The generic toast is a separate, real frontend bug: frontend/src/store/api.ts's APISubmitTagVote already throws {name, message} parsed from the backend's response body on any non-200, but every caller (AttributeChipPanel.tsx confirmed; likely also QueueTagQuestion.tsx/TagVotePicker.tsx/ ArtistVotePicker.tsx/NoMatchReasonStrip.tsx/PrintingTagPicker.tsx — same generic-string pattern, not yet individually confirmed) uses a bare .catch(() => {...}) that discards the thrown error and shows a hardcoded generic message regardless of what actually failed.

Shared rate-limit detail: post_submit_tag_vote, post_submit_printing_tag, and the artist-vote submission view all share one @ratelimit(...) budget (_printing_tag_rate_limit_key/ _printing_tag_rate_limit_rate in views.py, keyed by the client-generated anonymousId) — PRINTING_TAG_SUBMISSION_RATE (settings.py, 300/h as of 2026-07-17, was 20/h) covers a session's whole voting activity across all three endpoints, not a per-endpoint budget. A user mixing tag/printing/artist votes rapidly can trip it faster than "N tag votes alone" would suggest.

Fix: for the rate-limit class of failure specifically, nothing to fix server-side once the rate is sane for real usage (raised to 300/h alongside this entry — see settings.py's own comment for the full reasoning). The frontend's swallowed-error-message bug — noted here as still open — was fixed the same day by PR #47: every vote-submission .catch(...) (AttributeChipPanel.tsx, PrintingTagPicker.tsx, QueueTagQuestion.tsx, ArtistVotePicker.tsx, TagVotePicker.tsx, NoMatchReasonStrip.tsx) now surfaces the real error via errorToNotification/isRateLimited (common/apiErrors.ts), so a 429 reads as a friendly rate-limit message instead of a generic failure.

Refs: MPCAutofill/MPCAutofill/settings.py (rate + LOGGING comments), frontend/src/store/api.ts's APISubmitTagVote.

A single Jest test fails only in the full npx jest run, passes every time alone or with -t

Symptom: npx jest (default parallel workers) fails one specific test deterministically on every run, always at a waitFor/findBy* timeout — but npx jest path/to/file.test.tsx (whole file) and npx jest -t "the failing test name" (isolated) both pass reliably, every time, no code change in between.

Cause: this sandbox's CPU is shared across as many parallel Jest worker processes as npx jest defaults to spawning (one per detected core) — under that contention, a test whose passing path depends on a real (non-fake-timer) React state update landing inside the default 1000ms waitFor window can lose the race purely from scheduling delay, not from a logic bug. QuestionFeed.test.tsx's revealCard() helper (fires a synthetic animationEnd on RevealOverlay, since jsdom never runs the real CSS animation) hit exactly this: reliably reproduces the timeout under full-suite parallelism, reliably passes standalone or under --runInBand.

Fix: don't chase it as a logic bug once --runInBand (single worker, no contention) passes 3/3 — that's the confirming test, and matches the file's own comment about jsdom never firing animations for real. Where a helper's caller depends on the state update actually having landed (not just the event having fired), make the helper wait for its own effect (e.g. revealCard() now also asserts the overlay is gone via waitFor) rather than firing-and-hoping — this doesn't eliminate resource-contention timeouts entirely, but keeps the helper's contract honest. For a one-off local verification, prefer npx jest --runInBand over chasing the parallel-worker flake.

Refs: frontend/src/features/questionFeed/QuestionFeed.test.tsx's revealCard().

A Playwright click-through-navigation test fails locally on the first attempt but is genuinely green in CI

Symptom: npx playwright test (default retries: 0 locally, since playwright.config.ts only sets retries: 2 when CI is set) fails a real-browser click test deterministically — page.click() succeeds (correct single <a href>, no nested-anchor interception, click lands exactly on the target element per document.elementFromPoint) but expect(page).toHaveURL(...) times out, URL never changes. A [Fast Refresh] rebuilding console line lands right around the click. CI's own check-run for the same commit shows green, and its merged Playwright HTML report doesn't even surface the test by name for grepping (CI's reporter: "blob" prints no per-test lines to the job log at all — its silence isn't evidence either way).

Cause: Next.js dev mode (next dev, used by both local and CI Playwright runs per playwright.config.ts's webServer.command) compiles each page on first visit, not at server start. A test that navigates and clicks immediately can land its click while that first-visit compile/HMR cycle is still settling, interrupting the pending client-side next/link transition. This is a real, first- attempt flake, not a nested-anchor bug (rule that class out first via document.elementFromPoint/DOM inspection before spending time here) and not a mock/CI-status-lying situation.

Fix: don't chase it as an app bug once npx playwright test <file> --retries=2 (matching CI's own configured retry count exactly) shows the failing tests passing on retry, marked flaky rather than failed — that's the confirming test. CI's retries: 2 is an existing, deliberate project policy (not something to second-guess per-PR); a test passing via that policy is a legitimate CI green, not a masked failure. Don't try to verify a specific test's CI outcome by grepping job logs when the workflow uses reporter: "blob" — it prints nothing per-test regardless of pass/fail; download the merged playwright-report artifact (gh run download <run-id> -n playwright-report) if a real per-test read is needed, though its index.html is a JS-rendered SPA, not plain-text-greppable either.

Refs: frontend/tests/HomepagePanel.spec.ts, frontend/playwright.config.ts's retries/webServer.

test_valid_url[tappedout*]/[manastack] fails with InvalidURLException, unrelated to your change

Symptom: cardpicker/tests/test_integrations.py::TestMTGIntegration::test_valid_url[tappedout], [tappedout_with_www], or (as of 2026-07-22, PR #321) [manastack] fails with an InvalidURLException in CI on a PR that never touched the import-sites code (tappedout: #209, #213, #215; manastack: PR #321 - confirmed via direct curl that manastack.com/api/deck/list returns a genuine live 500, identical across two separate CI runs, not a one-off network blip).

Cause: that parametrize case makes a real HTTP request to the live site named in its Decks enum value — a genuine external-network dependency the test never declared. Whenever that site 503s, 500s, redirects, or is otherwise unreachable, ImportSite.request's default_is_response_valid check fails and raises, and the test goes red for a reason with nothing to do with the PR's diff. Every site in this parametrize (archidekt, cubecobra, magic-ville, manastack, scryfall, tappedout) is equally exposed to this in principle - tappedout and manastack are just the two that have actually been observed breaking so far, not the only two capable of it.

Fix: test_valid_url wraps the call in requests_mock.Mocker(real_http=True) and registers a mock response for each site once it's been observed flaking - tappedout.net/ www.tappedout.net (matched via TappedOut.get_host_names()) and, as of this fix, manastack.com (matched via ManaStack.get_host_names(), mocked with a JSON body shaped to match ManaStack.retrieve_card_list's own response_json["list"]["cards"] parsing so that code path still gets real coverage, not just a bypass) - while every other, not-yet-observed- flaking site in the same parametrize stays on the real_http=True fallback untouched. Chosen over a named skipif (the MOXFIELD_SECRET- gated pattern just above it in the same file) because there's no config flag to gate any of these on - only live reachability - and mocking keeps real parsing coverage instead of dropping it. If a THIRD site in this parametrize starts flaking in CI, the fix is the same pattern again: add one more mock.get(...) matching that site's get_host_names(), not a skip.

Refs: MPCAutofill/cardpicker/tests/test_integrations.py's test_valid_url, MPCAutofill/cardpicker/integrations/game/mtg.py's TappedOut/ManaStack, MPCAutofill/cardpicker/integrations/game/base.py's ImportSite.request.

test_rate_limited_after_exceeding_the_configured_rate fails intermittently in CI, passes on rerun, unrelated to your change

Symptom: TestPostSubmitTagVote::test_rate_limited_after_exceeding_the_configured_rate (cardpicker/tests/test_tag_votes.py) fails in CI (assert 200 == 429 - the test's SECOND request wasn't rate-limited as expected) on a PR that never touched rate-limiting, tag votes, or views.py (first observed PR #380, 2026-07-23). Re-running the SAME CI job with zero code changes passed clean (4m12s) - confirmed non-deterministic, not a real regression, before landing.

Cause: post_submit_printing_tag's own docstring already names the mechanism: django-ratelimit here "relies on Django's default (in-process) cache" - a single process-lifetime LocMemCache, not something pytest-django's per-test DB-transaction rollback resets. The sibling endpoint this specific test exercises (post_submit_tag_vote) shares the same in-process cache backend. Whichever OTHER test in the same worker process happens to run immediately before this one, and how many rate-limited requests it fires against an overlapping cache key/ window, can leave the sliding-window counter in a different state than a fresh run would see, so the outcome depends on execution order/ parallel-worker assignment, not just this test's own two requests. This is a pre-existing structural gap (no per-test cache clear), not something a single PR's diff can trigger or fix incidentally.

Fix applied so far: none - out of scope for a diff that doesn't touch rate-limiting; confirmed-flaky via a clean rerun and documented here instead of silently waved through, per this project's own "a red Backend-tests check now means something real - investigate it" rule (CLAUDE.md). If this starts recurring often enough to cost real review time, the real fix is a per-test cache clear (e.g. an autouse fixture calling django.core.cache.cache.clear()), the same category of fix test_valid_url's own entry above applies to network flakiness - not attempted here since one observed occurrence doesn't yet justify guessing at the right isolation boundary for every rate-limited endpoint in the same file.

Refs: MPCAutofill/cardpicker/tests/test_tag_votes.py's TestPostSubmitTagVote, and MPCAutofill/cardpicker/views.py's post_submit_printing_tag/post_submit_tag_vote @ratelimit decorator and _printing_tag_rate_limit_rate's own in-process-cache comment.

Every PR's prettier pre-commit check fails on a docs file the PR never touched

Symptom: the "Formatting and static type checking" CI job fails the prettier hook on docs/upstreaming/upstream-wiki-drift.md or docs/upstreaming/drift-log.md — files your branch's diff doesn't include (issue #214, surfaced by PR #213).

Cause: PR CI checks out the merge commit (branch + master), so any non-prettier-conformant content the weekly bot workflows (docs-upstream-wiki-drift.yml, upstream-drift-monitor.yml) committed straight to master rides along into every open PR's CI run and fails a file the PR never changed.

Fix: both auto-generated files are excluded from the prettier pre-commit hook via an exclude: pattern in .pre-commit-config.yaml — machine-generated weekly reports aren't hand-edited, so a hook gate on them protects nothing and only produces false reds. drift-log.md was also reformatted once so master itself started clean; the exclude is what keeps it that way regardless of what the bots commit next.

Refs: .pre-commit-config.yaml's prettier hook, .github/workflows/docs-upstream-wiki-drift.yml, .github/workflows/upstream-drift-monitor.yml.

Docs record a migration as "live on production" but showmigrations disagrees

Symptom: docs/infrastructure.md and docs/features/catalog-completion-plan.md recorded Stage C migrations 0068–0072 as applied to the persistent production Postgres (commit d1860257, issue #211); a separate PR's own notes additionally described migrations 0073–0075 as having auto-applied via a one-off docker compose run --rm django ... container's normal entrypoint migrate step. Neither matched the persistent DB's actual state: running manage.py showmigrations directly against the live containers, both before and after the 2026-07-20 django/worker rebuild, showed only 0068 applied beforehand — 0069–0075 landed only once the containers were recreated from a fresh master image.

Cause: "the migrate step should have run" is a plausible-sounding inference, not a verification step — it was narrated secondhand in a PR's own notes and then copied into docs as fact, never checked against the persistent DB directly. A one-off run --rm container's entrypoint genuinely does run migrate against whatever Postgres its compose resolution points at, but that's a claim about mechanism, not evidence about what actually happened on this specific run against this specific database.

Fix: before recording ANY migration as "live on production" in docs, run showmigrations (e.g. docker compose -f docker-compose.prod.yml exec django python manage.py showmigrations cardpicker) directly against the persistent containers and quote its real output — don't infer live status from "the migrate step should have applied it" reasoning, however mechanically plausible. Re-check immediately before and after any deploy/rebuild that's supposed to apply migrations, so a stale doc claim is caught by direct evidence rather than propagating silently through a chain of secondhand PR/doc narration.

A management-command test file is missing from the deployed prod container even though the command's own code fix is present

Symptom (found 2026-07-20, during the Stage C fetch/compute timing diagnostic — docs/reports/2026-07-20-fetch-compute-timing-diagnostic.md): cardpicker/tests/test_run_image_evidence_cohort.py (added by commit 0226a4de, the manager.shutdown()-ordering fix) does not exist anywhere under /MPCAutofill/MPCAutofill/cardpicker/tests/ inside the running mpcautofill_django container, confirmed via find. This is NOT simply "the container predates that commit" — the SAME commit's actual code fix (reading stop_event.is_set() into a local before manager.shutdown()) IS present in the container's own run_image_evidence_cohort.py, byte-verified via docker cp + diff against the current master checkout. So one file from a single commit is live in the deployed image and a second file from the identical commit is absent.

Ruled out, not confirmed: .dockerignore does not exclude tests/ (checked directly — only frontend/test-results and test-results are listed); the Dockerfile's COPY MPCAutofill /MPCAutofill/MPCAutofill copies the whole directory unconditionally, with no stage-specific exclusion of tests/ for the webserver/worker targets. Neither explains a single-file gap within one commit.

Cause: not determined this session — plausible candidates (an out-of-band hotfix of just the command file without a full rebuild; a build that ran from a checkout mid-commit; some other test-discovery quirk) were not investigated further, since this diagnostic's own actual verification need (does the new --profile code run correctly against a real prod cohort) was satisfiable directly via live --dry-run invocations instead of via this specific unit test. Left open rather than guessed at.

Workaround used this session: docker cp the test file (along with the two modified source files) into the running container before testing, then docker cp the pre-diagnostic originals back afterward (diff-verified clean) — see the report above for the full sequence. This is a workaround for verifying a specific change against a live container, not a fix for the underlying gap.

If you hit this again: before assuming a rebuild will restore parity, diff the deployed container's file tree against the exact commit git log says built it (get_baked_git_sha/GIT_SHA file, cardpicker.utils) file list, not just a spot-check of the files you happen to be touching — this gap was found by accident (checking whether a stub needed updating), not by a systematic audit, so other quietly-missing files may exist unnoticed.

A later "share/export this deck" design assumes a saved deck's DEK is stable, but it isn't

Symptom: implementing a feature that shares or otherwise re-uses a saved deck's existing DEK across time (e.g. a share link that's supposed to keep tracking a deck's live ciphertext) breaks the instant the deck is edited again — or, when designing such a feature, a spec's prose implies the deck's DEK only changes on an explicit action ("rotate"), never as a side effect of ordinary use.

Cause: encryptDeckPayloadForSave (frontend/src/features/savedDecks/deckPayload.ts) mints a FRESH DEK on every single call to saveDeck — including an ordinary content-editing "Update {name}" save of an already-saved deck, not just first-save (see SaveDeckModal.tsx's handleSubmit, which always calls encryptDeckPayloadForSave regardless of whether key is null). There is no code path that reuses an existing deck's DEK across saves. A design (this repo's own "PR-5, per-deck share links" spec included) can be written assuming the DEK is a stable, rarely-changing secret that only a deliberate "rotate" action touches — that assumption doesn't hold in this codebase and was never going to, once PR-4 shipped fresh-DEK-per-save.

Fix: don't build anything that expects a saved deck's DEK, or its wrapped form, to survive an ordinary edit-save unchanged. Instead, snapshot whatever needs to be independent of future edits (ciphertext, wrapped-DEK material, etc.) at the moment it's captured — exactly what SavedDeckKind.SNAPSHOT already does for the load-safety flow, and what SavedDeckShare (PR-5) does for share links: a frozen copy taken at creation time, not a live reference. See features/saved-decks.md's "Per-deck share links" section for the full writeup of this exact case.

psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "pg_type_typname_nsp_index" in django/worker startup logs during a rebuild

Symptom: a raw IntegrityError traceback (Postgres UniqueViolation on pg_type_typname_nsp_index, a catalog-level constraint on composite/enum type names, not anything defined in this repo's own models) appears in the django or worker container's startup log during migrate, immediately after a prod rebuild — looking identical, at a glance, to the "entrypoint migrate crash" failure mode documented above.

Cause: docker/django/entrypoint.sh runs migrate on both django and worker container startup (see "Entrypoint + migrate composition traps" above), and a prod rebuild starts both containers close together. When a migration creates a Postgres composite/enum type (anything that registers a row in pg_type), two concurrent migrate invocations can both pass Django's own "has this migration already applied" check before either commits, then both attempt the same type-creating DDL — one wins, the other raises this UniqueViolation. This is a benign startup race between the two containers' own migrate steps, not data loss or a corrupt migration.

How to confirm it's this and not real damage: check that the migration named in the traceback shows as applied (python manage.py showmigrations <app>) and that the table/type it creates has the expected schema (\d <table> in psql, or Model.objects.first() field-by-field) — if both check out, the losing container's migrate simply no-opped after the race and the winning one's write stands; no rerun or manual intervention needed. Confirmed exactly this outcome for 0076_saveddeckshare on 2026-07-20 (8 columns, schema correct, migration applied) immediately after seeing this traceback in a fresh rebuild's logs.

Fix: none applied — this is an accepted, self-healing race inherent to running migrate from two containers on the same startup, not a bug in the migration itself. If it starts blocking a container from reaching a healthy state (rather than just logging once and continuing), that would indicate a real regression and is worth revisiting; gating one of the two containers' migrate calls behind a lock, or having only one container run it, is a fix not yet implemented.

Adding one more always-visible Navbar.tsx link fails unrelated Playwright specs with "<a> from <nav> subtree intercepts pointer events"

Symptom: SavedDecks.spec.ts's Export/Import tests (and potentially other specs whose target element sits just below the fixed navbar) start failing in CI/Playwright with a page.waitForEvent("download") or locator.click timeout, and the retry log shows an unrelated navbar link (e.g. <a href="/editor" class="nav-link">) "intercepts pointer events" over the real target. The PR's own diff looks unrelated — it only added one new item to Navbar.tsx's left-hand Nav.

Cause: frontend/src/features/ui/Layout.tsx's ContentContainer and several other components (Explore.tsx, ProjectEditor.tsx, FinishedMyProject.tsx) hardcode a fixed pixel offset from NavbarHeight (frontend/src/common/constants.ts, = 50) assuming the navbar is always exactly one row tall. In the fully-authenticated, every-backend-feature-enabled state, the left-hand Nav's flex row is already near its horizontal wrapping budget — Bootstrap shrinks the flex items instead of moving them to a new line, so a long label ("What's That Card?") wraps internally and the whole fixed navbar renders taller than NavbarHeight (confirmed via page.locator("nav.navbar").boundingBox(): 64px tall on origin/master already, vs. the hardcoded 50px offset — a pre-existing, marginal, still-clickable overlap). Adding one more always-visible link to that same row pushes the real height further (88px), and now the entire target button sits under the taller bar rather than just its top few pixels, making it fully unclickable instead of merely graze-overlapped.

How to confirm it's this and not a real click-target bug in your own change: screenshot the affected page (page.screenshot()) and compare page.locator("nav.navbar").boundingBox() before/after your diff at the same viewport/auth state — if origin/master already shows the navbar's real height exceeding NavbarHeight's 50px (even by a little), your change didn't invent the bug, it just widened an existing crack.

Fix: don't add width to the already-crowded left-hand Nav when a new always-visible link is needed — put it in the right-hand ms-auto cluster instead (today just AuthWidget + the download-manager icon button + the Sources button, with real spare width). This keeps the navbar's real rendered height identical to origin/master's (confirmed via the same boundingBox() check) rather than papering over the symptom with a longer Playwright timeout or a spec edit.

Partial deeper fix landed (fix round on PR #305/#308's /whatsthat quiz-reveal hero, owner review) — frontend/src/common/useNavbarHeight.ts replaces the constant with a real ResizeObserver-measured value, but only for the two consumers a live report actually confirmed broken: Layout.tsx's ContentContainer (sitewide — every page's own top-of-content offset) and /whatsthat's own PageColumn height calc. Every other consumer (Explore.tsx, ProjectEditor.tsx, FinishedMyProject.tsx) still uses the static constant directly — this issue stays open for that broader swap. The hook also only measures the navbar's current height; it doesn't change anything about the underlying wrapping behaviour above (a crowded, every-link-visible state can still jump from 64px to 88px when one more link tips it to a second line) — the hook picks that jump up correctly via its own ResizeObserver, but the wrapping itself is still exactly the layout fragility this entry describes.

A reparse_collector_evidence/Stage D retraction pass silently never routes its own newly-touched cards to slow-path review

Symptom: manage.py local_calculate_verdicts --write runs cleanly over a cohort that was previously retracted/re-scanned (e.g. via reparse_collector_evidence --selector parser-bug/--selector no-text), casts real join-key votes/skips ([join-key] considered=N votes=...), and the gate passes — but CardScanLog.objects.filter(anonymous_id="stage-d-slow-path-v1", run_id=<this run>) comes back empty. The command's own [slow-path] log line for that invocation shows considered=0/routed=0 (or is easy to miss entirely if only the [join-key] line is being read) even though the same invocation's join-key stage just produced fresh no-match votes/no-hit skips that should qualify for routing.

Cause: reparse_collector_evidence.reparse_and_retract deletes a retracted card's stale stage-d-join-key-v1 CardPrintingTag/ CardScanLog rows before re-voting, but never touches that card's own stage-d-slow-path-v1 CardScanLog row from whichever ORIGINAL routing pass first flagged it. local_calculate_verdicts._slow_path_eligible_cards_queryset excludes any card that already carries a stage-d-slow-path-v1 row, unconditionally — this is the calculator's own idempotence/resume mechanism (never re-route a card twice), but it has no way to distinguish "already correctly routed under its current conclusion" from "routed once, under a conclusion that's since been retracted and replaced." A card retracted-and-revoted at the join-key layer is therefore silently excluded from ever being re-routed at the slow-path layer, even though local_calculate_verdicts --write correctly re-ran BOTH stages in the same invocation immediately afterward — the atomic combination worked exactly as designed, it just had nothing new to do for these specific cards.

How to confirm it's this: for the cards in question, check CardScanLog.objects.filter(anonymous_id="stage-d-slow-path-v1", card_id__in=<cohort>) — if a row exists with a run_id OLDER than the retraction that just ran, that's the stale marker; the calculator saw it and (correctly, per its own exclusion logic) didn't write a second one.

Practical read on severity, verified 2026-07-21: this is currently harmless for #262/#265's review-cluster backend specifically — cardpicker/review_clusters.py's _review_queue_card_ids() only checks for the EXISTENCE of a stage-d-slow-path-v1 row (any run_id), and its clustering signals are re-read fresh from each card's CURRENT ImageEvidence row at query time, never from anything stored on the stale CardScanLog row itself (which, additionally, has no per-card "why routed" field to be stale in the first place — skip_reason is hardcoded to the literal "to-review" for every row this calculator ever writes). A card in this state is therefore still fully visible and correctly clusterable today. The gap only becomes a real bug for a FUTURE consumer that reads something more specific from that row (e.g. a per-card routing reason, which doesn't exist yet).

Fix (spec'd, not yet built — see docs/features/catalog-completion-plan.md's "Recovery-arc lessons" section): extend reparse_and_retract to also delete the retracted card's own stage-d-slow-path-v1 CardScanLog row in the same pass it deletes the stage-d-join-key-v1 rows, mirroring the existing delete and reusing the same safety gate. Until that ships, treat any retraction pass as needing a manual check of whether its cohort also needs its slow-path marker cleared before the next local_calculate_verdicts --write can actually re-route it under its new conclusion.

Playwright tests behave like a 1280×720 desktop viewport even though playwright.config.ts sets 800×600

Symptom: a change that's supposed to be invisible/behave differently below some breakpoint (a responsive drawer, a collapsed toolbar, a media-query-gated style) shows up as always-inline/always-expanded in every existing test in the suite, even though the chromium project's use block clearly declares contextOptions: { viewport: { width: 800, height: 600 } }.

Cause: contextOptions is not a real Playwright TestOptions field — Playwright's actual browser-context config lives at the top level of use (viewport, reducedMotion, etc. directly), not nested under a contextOptions key. playwright.config.ts's chromium project spreads ...devices["Desktop Chrome"] first (which sets a top-level viewport: {1280, 720} and reducedMotion is absent entirely), then adds a sibling contextOptions: {...} object that Playwright silently ignores — so every test in the repo has actually been running at Desktop Chrome's stock 1280×720, full-motion, the whole time. Confirmed by evaluating the real rendered viewport/computed style inside a test (page.evaluate(() => window.innerWidth) and inspecting an Offcanvas's actual class list) rather than trusting the config file's stated intent.

Fix: for a test that genuinely needs a narrower/specific viewport (or reducedMotion), use test.use({ viewport: {...}, reducedMotion: "reduce" }) at the top level of a test.describe block (or per-test) — that field name IS real and reliably overrides the project default for just that scope, unaffected by the dead contextOptions wrapper. Don't "fix" the stale 800×600 intent in the project config itself as a drive-by — every existing spec in the repo was authored and passing against the actual 1280×720 desktop viewport, so correcting the config to match its stated intent would silently change the effective breakpoint tier (and therefore behavior) of the entire existing suite in one line, far outside whatever single feature change prompted noticing this. Filed during issue #266 (mobile /display responsive shell, frontend/tests/DisplayPage.spec.ts's phone-viewport describe block).

A ResizeObserver-driven layout value is "stuck" at its initial default in one CI shard/spec but correct everywhere you check it manually

Symptom: a value derived from a ResizeObserver (e.g. a measured container width feeding a child's render size) intermittently renders as its unclamped, un-narrowed default rather than the real, smaller, currently-available size — causing that child to overflow its own flex column and visually spill under/over a sibling. Manual verification (screenshots, a scratch Playwright script hitting the same page) shows the correct, narrow value every time; only a specific CI shard, or a specific other spec file exercising the same page more deeply (clicking further into nested UI before checking), reproduces it. Playwright reports the interaction failure as an unrelated element "intercepts pointer events" — the real target is exactly where expected, but something else (the wrongly-sized sibling) paints on top of it at that screen position.

Cause: an observer wired up via the "lazy-ref-initialization" pattern (if (ref.current == null) ref.current = new ResizeObserver(...), directly in a component's render body — a legitimate React pattern for an expensive object that should exist exactly once, but not StrictMode-safe for one that also needs observe()/disconnect() called on it from a callback ref) can end up with more than one live instance simultaneously in next dev's reactStrictMode: true (double- invoke mount/cleanup/mount in development only). Each instance calls the same state setter independently; whichever instance's callback fires last wins, and if a stale/duplicate instance is still attached to (or re-observing) a node whose size hasn't yet settled to its final, flex-constrained value, its late-firing callback silently overwrites the correct measurement with a stale, too-large one. This is a genuinely timing-dependent race — it doesn't reproduce every run, and a single manual check right after the interaction you expect to trigger it can easily land on the "correct" side of the race, which is why it slipped through this change's own pre-push manual/screenshot verification and only showed up as a real CI shard failure in a different spec file that happened to interact more deeply (and therefore add more time/render passes) before checking the result.

Fix: don't hand-roll observe()/disconnect() calls against a lazily-constructed single observer instance from a plain callback ref. Use the standard React pattern instead: a callback ref that writes the DOM node into useState, plus a useEffect keyed on that state value that creates a fresh ResizeObserver scoped to the current node and returns its own disconnect() as the cleanup function. This makes setup/cleanup pairing explicit and StrictMode's double-invoke mount→cleanup→mount cycle exercise it correctly every time — there is never more than one live observer racing to set state, regardless of how many times the effect re-runs. Confirmed via page.addInitScript() wrapping window.ResizeObserver to log every constructed/observe/ disconnect/fired: <width> call — the lazy-ref version showed multiple constructed instances and a final stale fired: 960 (the unclamped default) even though the container was genuinely ~488px wide by then; the state+effect version settled on one instance and the correct value. Filed during issue #266 (frontend/src/features/display/DisplayPage.tsx's sheet-region fit-to-width ResizeObserver, caught by tests/SelectVersionSection.spec.ts failing in CI shard 4/4 only, not in the 39 tests run locally pre-push).

Running backend pytest on the production box without touching the live docker-compose.prod.yml stack

Symptom: you're on the production Oracle-box machine (not a cloud/web session), need to run backend tests for a small fix, and the only running Django/Postgres/Elasticsearch containers are the live, traffic-serving mpcautofill_* ones from docker-compose.prod.yml — rebuilding or exec-ing into them to run a test suite risks disrupting production, and there's no obvious per-worktree isolated stack (container names are fixed machine-wide, not per-worktree).

Cause: MPCAutofill/manage.py/pytest need real Postgres + Elasticsearch to run against, but this box's only running instances are production's own. It's easy to assume you have to spin up (or touch) the prod compose stack to get there.

Fix: you don't need to touch the prod containers at all. A pre-provisioned host venv already exists at /home/ubuntu/.venvs/mpcautofill-pilot with Django/pytest/pytest-django/ tesseract/elasticsearch-dsl already installed — check for it before creating a fresh one. MPCAutofill/MPCAutofill/settings.py's own DATABASE_HOST/ELASTICSEARCH_HOST env-var defaults are already localhost, and docker-compose.prod.yml already exposes Postgres on 127.0.0.1:5432 and Elasticsearch on 127.0.0.1:9200 — so /home/ubuntu/.venvs/mpcautofill-pilot/bin/python -m pytest run directly from MPCAutofill/ on the host connects to the live containers' exposed ports with zero env overrides needed. This is safe: pytest-django creates its own ephemeral test_* database via CREATE DATABASE for the run and tears it down after (the standard pytest-django lifecycle) — it never reads or writes the actual production mpcautofill database/index. No docker rebuild, no docker compose exec, no risk to the live stack. Verified 2026-07-21 running cardpicker/tests/test_local_ocr.py, test_local_identify_printing_tags.py, test_image_evidence.py, test_golden_set.py, test_local_calculate_verdicts.py, and test_reparse_collector_evidence.py together (370 passed) this way against the live prod containers with no observed impact.

Test containers no longer take fixed host ports — Bind for 0.0.0.0:9300 failed: port is already allocated is FIXED (2026-07-29)

Status: FIXED. Concurrent pytest cardpicker/tests/ runs on one box no longer collide over host ports, and there is no workaround left to apply. This entry is kept because the failure it describes was mistaken for a code regression three separate times in one day, so the symptom needs to stay searchable — and because a run against an OLD branch (anything based before this fix) still behaves the old way.

Symptom (historical, pre-fix): a full pytest cardpicker -q run (not a targeted file/module) produced hundreds of ERRORs spread across many files that had nothing to do with your change (test_views.py, test_vote_consensus.py, test_sources.py, etc.) — docker.errors.APIError: 500 Server Error for http+docker://localhost/... (Bind for 0.0.0.0:9300 failed: port is already allocated, or the same for :47000). Every test in an affected run showed as ERROR, not FAILED, and the traceback bottomed out in cardpicker/tests/conftest.py's postgres_container/elasticsearch_container fixtures rather than in your own code; running the files your change actually touched passed cleanly, which was the tell.

Cause (historical): the session-scoped testcontainers Postgres and Elasticsearch containers in cardpicker/tests/conftest.py were pinned to fixed host ports (POSTGRES_PORT = 47000, ELASTICSEARCH_PORT = 9300) via with_bind_ports. This machine runs more than one worktree session at a time (see WORKERS.md at the repo root, machine-local) sharing one Docker daemon, so the second session to start lost the bind. An intermediate fix made those ports env-overridable (TEST_POSTGRES_PORT/TEST_ELASTICSEARCH_PORT) but nothing ASSIGNED distinct values, so the DEFAULT still collided — overridability without allocation does not solve concurrency, and every session still had to hand-pick an offset it hoped nobody else had taken.

How the ports are allocated now: conftest.py starts both containers with no host binding at all — only the with_exposed_ports their testcontainers constructors already set — so Docker assigns a free ephemeral host port per container, and the fixtures read back what it actually assigned with get_exposed_port(). Two session fixtures, postgres_port and elasticsearch_port, publish those values; django_db_modify_db_settings and the elasticsearch fixture point Django's DATABASES["default"]["PORT"] and ELASTICSEARCH_DSL at them, and the pytest-elasticsearch plugin's own elasticsearch_port option is threaded to the same value so elasticsearch_nooproc cannot fall back to a stale hardcoded 9300. Nothing wants a specific port, so nothing can collide — this is a closed race, not a narrowed window (picking a random constant at import time would only be the latter). Isolation is unchanged: each run still gets its own containers, its own test_* database and its own index.

TEST_POSTGRES_PORT / TEST_ELASTICSEARCH_PORT still work and still pin a deterministic host port — use them when you need to attach psql or an Elasticsearch client to a live test container, or want a reproducible port in a debugging session. Setting one re-introduces the collision risk for that run by design; it is opt-in, and each variable is independent (unset means ephemeral for that container only). test_harness_isolation.py pins all of this — including that the DEFAULT requests no host binding — so the fix cannot rot back into a fixed port silently.

What is NOT fixed by this: general resource contention on a shared box. Several full suites at once still compete for CPU, memory and the Docker daemon, and can still produce django.db.utils.OperationalError: connection to server at "localhost" from Postgres connection pressure rather than a port bind. If you see mass ERRORs with that message (and no port is already allocated), check for concurrent pytest processes (ps aux | grep pytest) and containers (docker ps) before debugging your own change. Orphaned containers from an interrupted prior run no longer block anything — they hold only a port nobody wants — but they do still consume memory, so docker rm -f is worth doing on containers you recognise as your own leftovers. Never remove one you don't recognise: it may belong to a live session mid-test.

Two concurrent frontend E2E runs — the second one tests the FIRST one's code, then dies mid-run (FIXED 2026-07-29)

Status: FIXED for the default path. Kept searchable because the symptom is indistinguishable from a real regression, and because a run against a branch based before this fix still behaves the old way.

Symptom (historical, pre-fix): you start npx playwright test (or npm run test-e2e) in one worktree while another worktree's E2E run is already going. Your run's first specs pass, then a block of unrelated specs fails all at once — typically page.goto failures out of tests/test-utils.ts's loadPageWithDefaultBackend, or assertions about UI that your branch did not touch. Reproduced on unmodified master (2026-07-29): two overlapping runs of the same four spec files, the second started three seconds after the first — the first passed 1/1, the second failed 8 of 10 with navigation errors, and every failure pointed at application code rather than at the port.

Cause (historical): frontend/playwright.config.ts hardcoded http://localhost:3000 in both use.baseURL and webServer.url. Off CI (reuseExistingServer: !process.env.CI) Playwright does not start a second dev server when something already answers on that URL — it reuses it. So the second run silently pointed at the first run's next dev, i.e. tested the first worktree's checkout, and then lost the server outright the moment the first run finished and tore it down. Same bug class as the fixed test-container host ports in the entry above (#571), with a nastier failure mode: containers failed loudly at bind time, this one produced quiet wrong-tree results first.

How the port is chosen now: playwright.config.ts's resolvePort() asks the kernel for a free port (bind port 0, read the assignment back, release) unless PLAYWRIGHT_PORT is set, and exports the result into the environment so Playwright's worker processes — which each re-load the config in their own process — inherit the same port instead of drawing their own. baseURL, webServer.url and the webServer command all derive from that one value, and playwright.perf.config.ts inherits it by spreading the base config. Verified 2026-07-29 by re-running the exact scenario above against the fix: two next-server processes listening on different ports for the whole overlap, 10/10 and 10/10 passing, four workers each.

Set PLAYWRIGHT_PORT when you want a fixed port. PLAYWRIGHT_PORT=3000 restores the old behaviour, including reusing a npm run dev you already have running — worth it for a fast edit-run loop, since it skips the dev-server boot on every invocation. Any pinned value re-introduces the collision for that run, by design.

What is NOT closed by this — read before assuming a port can never collide again:

• The kernel's assignment is released before next dev binds it, and that gap spans npm run dev plus Next's boot (seconds, not microseconds). Playwright's webServer.url has no port-0 read-back equivalent, so unlike #571 nothing holds the binding across the gap. A foreign process taking the port inside that window is possible. What it can no longer do is pass wrongly: next dev is invoked with an explicit --port, and Next only walks to the next free port when it chose the port itself — given one explicitly it exits with EADDRINUSE, so the run aborts at webServer startup with Error: Process from config.webServer exited early. If you see that, just re-run; you drew an unlucky port.
• Two runs in the SAME directory still collide, port or no port. Both next dev servers write frontend/.next, and Playwright writes playwright/.auth/, test-results/ and the HTML report there too. Concurrency is safe across worktrees, which is how this box actually runs; it is not safe across two shells cd'd to one checkout.
• General resource contention is unchanged — several dev servers plus browser pools at once still compete for CPU and memory, and can time out on their own.

A per-instance viewBox crop on an inlined SVG shows the entire source art instead of just its own band

Symptom: three separate <svg viewBox="..."> elements, each meant to crop a different horizontal band out of the same inlined wordmark path data (WhatsThatWords.tsx, issue #305), all render the full, uncropped wordmark at slightly different sizes instead of their own distinct slice — confirmed via getAttribute("viewBox") in a live page that each element's viewBox attribute IS correct, and overflow: hidden is already set and computes correctly too, yet the bug persists.

Cause: two independent CSS behaviors compound here, neither obviously connected to "SVG cropping" on its own:

1. A root-level <svg> (i.e. one that isn't nested inside another <svg> in the DOM) can still be affected by a Flexbox ancestor's default align-items: stretch — if the flex container is flex-direction: column, stretch operates on the CROSS axis, which for a column flex is WIDTH. A replaced element (SVG counts) with CSS width: auto is exactly the trigger condition for stretch to override its own intrinsic (height × viewBox-aspect-ratio-derived) width and force it to the full container width instead.
2. Once the SVG's rendered box is stretched far wider than its own viewBox's aspect ratio, preserveAspectRatio's default xMidYMid meet recomputes its internal scale against that stretched box — and at a large enough width/height mismatch, more of the underlying artwork becomes visible within the (correctly, overflow: hidden-clipped) box than the viewBox rectangle alone would suggest, because the box itself grew, not because the crop stopped applying. overflow: hidden and a correct viewBox attribute are both real and both necessary, but neither one controls the SVG's own rendered box size — that's ordinary CSS layout, upstream of either.

Fix: stop the stretch at the source — add align-items: flex-start (or center, matching whatever horizontal alignment is wanted) to the flex-column parent, so each SVG child keeps its own intrinsic, height-derived width instead of being forced to the container's full width. Diagnosed by isolating the exact repro in a minimal throwaway HTML page (one flex-column parent, three stacked SVGs, no React/Next in the loop at all) rather than debugging inside the full app — confirmed the same three-line reproduction failed identically, and that removing only the flex-column wrapper (testing the single SVG alone) fixed it, which is what pointed at the flex cross-axis stretch specifically rather than the SVG/viewBox mechanics themselves.

/display's floating "n/M" sheet-position pill under-reports the last sheet (shows 2/3 instead of 3/3) after scrolling all the way down

Symptom: DisplayPage.spec.ts's sheet-position-pill test (test title carries a trailing "(D17)" — the test's own name literally quotes the retired label from proposal-h-display-layout-spec.md's sheet-presentation refinement decision, kept verbatim here since it's a direct quote of the code, not a fresh reference) — "the floating sheet-position pill updates live while scrolling at phone width" — fails with getByTestId('display-sheet-position-indicator') stuck at "2/3" after scrolling the last sheet into view via scrollIntoView({ block: "center" }) at 390px wide — reproduces 100% of the time in isolation, both locally and in CI (not the intermittent kind), even though the same test was merely marked "flaky" (2 real failures then a lucky pass) in the PR that introduced it (#313).

Cause: the indicator's IntersectionObserver (DisplayPage.tsx, visibleSheetIndex) decides "which sheet is current" purely by a thin vertical center band (rootMargin: "-45% 0px -45% 0px") against the page viewport. That heuristic structurally can't ever select the FIRST or LAST sheet once the scrollable content-container (Layout.tsx) is already at its true scroll extreme and a boundary sheet is short enough that there's no room left below/above it to move its own center through that band — confirmed by measuring the container directly: scrollHeight - clientHeight (the real max scroll) was ~250px, but centering the last (short) sheet would have needed roughly 500px+ of scroll, a shortfall of 250-280px that no amount of further scrolling can close, because the container is already at its true bottom. This isn't specific to any one card count or viewport — it's inherent to a fixed center-band test whenever a boundary sheet is short relative to the viewport, and is unrelated to Footer/Navbar sizing (/display doesn't even render a Footer).

Fix: inside the same IntersectionObserver callback, read the scroll container's own scrollTop/scrollHeight/clientHeight (via entries[0].target.closest('[data-testid="content-container"]')) and check the true scroll extremes FIRST — at scrollTop <= EPSILON, force index 0; at scrollTop + clientHeight >= scrollHeight - EPSILON, force the last index — falling through to the existing center-band Math.min(...intersectingIndices) logic only when not at an edge. Kept as a single writer to visibleSheetIndex (not a second scroll listener racing the observer) since the observer already re-fires on the same settling scroll event that changes intersection state. Verified 5/5 clean repeats of the sheet-position-pill test plus the full 28-test DisplayPage.spec.ts suite and CI's own shard-2/4 (76 tests) locally.

run_image_evidence_cohort (Stage C) parent process's RSS climbs unboundedly and OOMs the whole box on a long run

Symptom: a long (tens-of-thousands-of-cards) run_image_evidence_cohort invocation shows parent-process RSS climbing steadily over the run's lifetime — not a leveling-off/plateau — reaching tens of GB and either OOM-killing the whole box or (if a watchdog is in place) getting stopped partway through. Arithmetic: retained bytes ÷ cards processed lands around 250-350KB/card, which is the size of one raw fetched card image buffer, not anything decoded or persisted.

Cause: the decoupled fetch/compute driver (_run_cohort, added by the Stage C fetch/compute decoupling — #228/#235, docs/features/catalog-completion-plan.md's "Stage C: fetch/compute decoupling" section) submitted every card in the cohort to the fetch thread pool UP FRONT, and only gated COMPUTE submission behind --queue-depth. The design doc's own memory-budget arithmetic for that section silently assumed fetch completion was ALSO bounded by that window — the implementation never enforced it. Fetch (I/O-bound, paced by a 6-way concurrency limiter) completes cards faster than the CPU-bound compute stage consumes them, so fetch raced arbitrarily far ahead of consumption over a multi-hour run, accumulating raw image buffers for the entire fetched-but-not-yet-consumed backlog — in the worst case, most of the remaining cohort. A secondary, much smaller (~1% of the effect) contributor: the fetch_futures collection tracking submitted futures was never pruned as futures were consumed, so even already-handled results stayed referenced (and their retained image bytes stayed resident) for the rest of the run.

Fix (2026-07-22): fetch submission is now drip-fed (_submit_more_fetch), bounded by the same --queue-depth knob already used to gate compute submission, so total outstanding fetch-stage work (in flight + completed-but-not-yet-consumed) never exceeds that window regardless of cohort size. The fetch_futures tracking set now discards each future the instant its result is consumed. The command also logs the parent's own RSS (from /proc/self/status) on every progress line — so the next time this (or anything like it) accumulates unexpectedly, the log itself shows it climbing — and a new --max-rss-mb flag (off by default) turns crossing a threshold into a clean, resumable stop instead of relying on the OS OOM-killer; the command's own resume filter (skip cards whose ImageEvidence row already carries every manifest extractor's version key) makes a re-invocation after any stop safe. Diagnosed via a synthetic repro (tracked, weakref-observable payload objects standing in for real fetched bytes, no live prod run) that reproduced the exact growth-with-cohort-size pattern before the fix and confirmed the bound held after it — see cardpicker/tests/test_run_image_evidence_cohort.py's TestRunCohortFetchMemoryBound for the same property turned into a permanent regression guard.

guard_master.py denies a git merge/git push you ran from inside a feature worktree

Symptom: a command like cd /path/to/feature-worktree && git merge origin/master (resolving a feature branch's conflicts against master) gets denied with [guard_master] git merge into master is owner-only, always, even though the branch actually being merged into isn't master.

Cause: guard_master.py's git merge/git push rules judged branch state via current_branch(cwd), where cwd is the session's registered working directory — not wherever the command's own cd <path> && chain actually pointed git at. A session whose registered cwd happens to be a master checkout gets every cd <other-dir> && git merge ... it runs judged against master, regardless of what's actually checked out at <other-dir>. Confirmed in production 2026-07-22: this wrongly blocked legitimate feature-branch conflict resolution.

Fix (2026-07-22): added effective_dir(command, session_cwd, target_re) — splits the command into simple-command segments on ;, &, |, &&, and ||, locates the segment matching the merge/push the calling rule already detected, then walks backward from it collecting only the segments joined to it by an unbroken chain of && (a cd behind a ;, single &, |, or || boundary is a different shell statement and is deliberately not followed — real && semantics mean only a genuinely chained cd can be trusted to have actually run before the merge/push did). Processes that chain applying each whole-segment cd <path> in turn; falls back to session_cwd (today's pre-fix behavior) whenever there's no leading cd chain, or any cd in it targets a path that isn't a real directory — a resolution failure is never more permissive than session-cwd-only judging. Used in both the git merge and git push worktree rules' current_branch(...) == "master" checks. Unchanged: the unconditional gh pr merge denial, the --ff-only exemption, all deny messages, log_stub's behavior, and the push rule's in_worker_worktree gate (still computed from raw session cwd, since it identifies which session is running, not which directory a given command targets). See .claude/hooks/test_guard_master.py for the regression cases.

Same-day tightening: the first cut of this fix also resolved a bare git -C <path> anywhere in the command via an unanchored scan. That turned out to be a false-ALLOW risk in the opposite, more dangerous direction: an earlier, unrelated git -C /some/repo status && git merge origin/master could get its unrelated -C path substituted in for the real merge's own context, wrongly ALLOWing a genuine merge-into-master. Confirmed by direct trace, then closed by dropping git -C support entirely rather than anchoring it — effective_dir() now only follows cd chains. This reopens a narrower, safe gap: a bare git -C <path> merge/push ... command isn't recognized as a merge/push attempt at all today, since the calling rules' own detection regexes require "git" and "merge"/"push" adjacent with no intervening flag — accepted, since under-triggering only ever produces an unnecessary DENY, never a wrong ALLOW, and cd-chains are the pattern that actually occurred in production.

2026-07-24: extended to Edit/Write/NotebookEdit, closing the worktree-path trap at the tool layer. Everything above in this entry covers guard_master.py's Bash-matched rules only. Separately from those, .claude/settings.json's PreToolUse matcher now also fires on Edit, Write, and NotebookEdit, and guard_master.py dispatches those three to check_worktree_write_guard() before falling through to the Bash-only logic. Motivation: four independent sessions accidentally edited the shared main checkout via absolute-path Read/Edit/Write calls on 2026-07-23/24 (all self-caught before landing anything) — the exact failure mode described in docs/lessons.md's "Absolute paths to the repo root silently target the wrong checkout in a worktree session", but via a file tool instead of git/gh in Bash, so the existing rules never saw it.

Mechanics: worktree_main_checkout_root(cwd) returns the path before /.claude/worktrees/ in the session's cwd (None if the session isn't a worker worktree at all, in which case this rule is a no-op). resolve_write_target(tool_input, cwd) pulls the target path out of tool_input["file_path"] (Edit/Write) or tool_input["notebook_path"] (NotebookEdit), joining a relative value against cwd first. targets_main_checkout(target, main_root) is True only when the resolved target is under main_root but NOT under main_root's own .claude/worktrees/ subtree (so the session's own worktree, any other session's worktree, and anything outside the repo entirely — /tmp, the orchestration repo, the memory dir — all pass through untouched), with one deliberate exception: WORKERS.md and journal/ (see the docs/lessons.md entry above) are excluded from the block, since a worker worktree session writing its own coordination row there is the documented, intentional workflow, not the trap.

Read is deliberately never gated — blocking reads would break legitimate cross-referencing against the main checkout's on-disk state (e.g. diffing a worktree's change against what's on master), and a read can't silently land content on the wrong branch the way a write can.

Regression coverage: .claude/hooks/test_guard_master.py's write_guard_cases (end-to-end, via run_hook_payload(), against the real tool_input shapes each of the three tools sends) plus direct unit cases against the three helper functions above. This suite has no CI wiring (same as the pre-existing Bash-rule tests it extends) — it's a local dev-loop check, run manually with python3 .claude/hooks/test_guard_master.py.

DisplayPage.spec.ts's "floating sheet-position pill updates live while scrolling at phone width (D17)" fails intermittently with "2/3" instead of "3/3"

Symptom: tests/DisplayPage.spec.ts's sheet-position-pill test (phone viewport, 18 cards / 3 sheets) intermittently reports the display-sheet-position-indicator still reading 2/3 after scrollIntoView({block:"center"}) on the last sheet, where 3/3 is expected - reproduces even running that ONE test alone, --repeat-each=5+, on both a freshly-modified branch and on plain origin/master with zero changes.

Cause: a genuine, pre-existing flake in this test's own IntersectionObserver-based timing (its rootMargin: "-45% 0px -45% 0px" thin center-band check races the browser's scrollIntoView completing), not a regression from any particular change - confirmed by reproducing the same ~1-in-6 failure rate on unmodified origin/master in this same sandboxed VM. A change that happens to add or remove incidental rendering cost nearby (e.g. widening a useMemo's dependency array on the sheet-content builder) can shift the failure rate up or down without being the actual cause - don't chase a "regression" here without first checking the identical test against a clean origin/master checkout in the same environment, --repeat-each=5 or more (a single passing/failing run either way is not enough evidence).

Fix/mitigation: none applied - this is a test-timing flake to tolerate (retry) rather than a product bug to fix; if it starts failing CI at a rate that matters, the real fix is loosening the test's own scrollIntoView-then-assert race (e.g. an explicit waitForFunction/poll on the indicator text before asserting, instead of relying on expect(...).toContainText's own retry window alone), not touching the sheet-content pipeline it happens to render.

A styled-component CSS template literal throws confusing TS1005/TS1351/TS1443 parse errors a few lines below a comment that "looks fine"

Symptom: adding an explanatory // comment INSIDE an emotion/styled-components CSS template literal (between the opening styled.div` and its own closing backtick) produces a cascade of unrelated-looking TypeScript parse errors - TS1005: ',' expected, TS1351: An identifier or keyword cannot immediately follow a numeric literal, TS1443: Module declaration names may only use ' or " quoted strings - pointing at code several lines AFTER the comment, not at the comment itself.

Cause: the comment contains a literal backtick character (e.g. writing `minmax(0, 7.5rem)` or `position: sticky` inline to mark it as code). A JS/TS template literal has no concept of a "CSS comment" - stylis (the CSS preprocessor emotion/styled-components run template-literal content through) treats //... as a comment, but that processing happens AFTER the JS parser has already tokenized the template literal as a plain string. The JS parser itself doesn't know or care that stylis will later treat some of this text as a comment - it just scans for the next unescaped backtick to close the string, or ${ to start an interpolation. A literal backtick anywhere inside - even inside what stylis would consider a // comment - closes the JS template literal early, and everything after that point (until the NEXT stray backtick, which reopens ANOTHER unintended template literal) gets parsed as ordinary TypeScript code instead of CSS-in-JS string content, producing parse errors that land wherever that reopened/misparsed region happens to contain something syntactically invalid.

Fix: never use a literal backtick inside a comment that lives between a styled-component's own opening/closing backticks - drop the backticks around the quoted CSS/prop name entirely (plain text reads fine: minmax(0, 7.5rem) track instead of `minmax(0, 7.5rem)` track), or move the comment to sit OUTSIDE the template literal (a regular // comment directly above the const X = styled.div line declaration has no such restriction - backticks there are just ordinary text in a real JS single-line comment, not inside a string at all). npx tsc --noEmit catches this immediately and precisely once you know to look for a stray backtick upstream of the reported line - the error location itself is not where the actual mistake is.

getWorkerImageURL/other NEXT_PUBLIC_*-reading helpers return undefined when called directly from Playwright TEST code (not the app)

Symptom: a Playwright spec imports a shared helper that reads a NEXT_PUBLIC_* env var (e.g. common/image.ts's getWorkerImageURL, used to compute an expected URL for page.route() interception) and calls it directly in the test body - it returns undefined even though playwright.config.ts's webServer.env clearly sets that exact variable, and the app's own BROWSER-rendered <img src> resolves to a real, correct URL built from the same helper.

Cause: NEXT_PUBLIC_* variables are inlined into the BROWSER bundle by Next.js's webpack build step at the point the dev server starts (playwright.config.ts's webServer.command: "npm run dev" spawns that process with env scoped to just that child process) - they are not, and were never meant to be, environment variables available to arbitrary Node code. The Playwright TEST RUNNER itself (npx playwright test) is a completely separate Node process with no bundling step of its own, so process.env.NEXT_PUBLIC_IMAGE_WORKER_URL (or any other NEXT_PUBLIC_* var) is simply unset there regardless of what webServer.env configures for the spawned dev server.

Fix: if a test genuinely needs to compute the same URL production code would build (e.g. to page.route()-intercept it, rather than guessing/hardcoding a pattern that can drift out of sync with the real implementation), set process.env.<THE_VAR> explicitly at the top of that test body to mirror playwright.config.ts's own webServer.env value, immediately before calling the helper - this only affects the Node-side computation in the test, not the already-running browser (which resolved its own copy independently, at its own build time), so it's safe and has no effect beyond that one calculation. Also watch for page.route()'s string argument being a GLOB pattern, not a literal string - a computed CDN URL with a ?jpgQuality=100-style query suffix needs its ? escaped (or the whole pattern passed as a RegExp instead), since glob ? means "exactly one arbitrary character," not literal query-string syntax.

docker exec mpcautofill_django ps → "executable file not found"

Symptom: sudo docker exec mpcautofill_django ps aux (or any other ps invocation inside the running Django container) fails immediately with something like OCI runtime exec failed: exec: "ps": executable file not found in $PATH.

Cause: the container's base image doesn't ship a ps binary at all — this isn't a $PATH misconfiguration, ps genuinely isn't installed.

Fix: use the compose CLI's own process view instead of shelling in — sudo docker compose -f docker-compose.prod.yml top <service> (e.g. top django) reads process info from the host's own view of the container, no in-container binary required. docker compose v2 with the space, per this repo's own tooling convention — never the hyphenated docker-compose v1 binary.

Logged out of admin/moderation after a backend deploy

Symptom: an owner/moderator session that was logged in via Discord OAuth (or Django admin) stops being authenticated right after a backend deploy that recreates the mpcautofill_django container — same browser, same cookies present, just no longer treated as logged in.

Cause: unconfirmed as of this writing — a container recreate does invalidate the session, but which layer actually breaks it (an in-memory/process-local session store losing state on restart, vs. an OAuth token the recreate somehow invalidates) has not been isolated. Don't assert a specific mechanism until it has actually been traced through a recreate.

Fix: log back in; no workaround needed beyond that. Next occurrence: before writing this off as "expected," reproduce deliberately (recreate the container, watch whether the session survives) and confirm which layer is actually responsible, then replace this entry's cause with the confirmed one.

Any ?server=... link that also carries another query param or a URL fragment silently loses everything but server on load

Symptom: visiting e.g. /editor?server=http://host:8000&foo=bar#frag (or any other page) ends up at plain /editor in the address bar once the page finishes mounting — not just foo/frag dropped, server itself vanishes too, along with a console warning: Unknown key passed via urlObject into url.format: server. Confirmed to reproduce on a direct page load with no redirect involved at all — this is not specific to any one route.

Cause: useBackendSetter.ts's own ?server= clean-up step calls router.replace({ server }, undefined, { shallow: true }) — passing a bare { server } object with no pathname as the url argument. Next.js's url.format doesn't recognize server as a valid key on an object missing pathname/query, silently drops it, and the resulting URL is just the bare pathname. Reproduced during the Proposal H route-swap task (2026-07-23) while verifying /display's new query-param-preserving redirect to /editor — the redirect itself (pages/display.tsx, reading window.location.search/hash directly) forwards params/fragment correctly on its own (verified with a ?server=-free URL, which never triggers this code path); this bug fires independently, on /editor directly, once useBackendSetter's effect runs.

Fix: not fixed as part of the route-swap task (unrelated, pre-existing, and in practice harmless for real traffic — no code in the app builds a /display or /editor link with extra query params/a fragment today, per a repo-wide grep). The real fix is straightforward whenever someone picks it up: pass a proper { pathname: router.pathname, query: { ...router.query, server } } shape (or drop the shallow URL-object call entirely in favor of router.replace(router.asPath.split("?")[0] + buildQueryString(...))) so existing query params and the fragment survive the same clean-up step.

chunkErrorRecovery.spec.ts's expect.poll(() => reloadRequests).toBe(1) fails with "Received: 0" in CI, passes locally

Symptom: one of the chunkErrorRecovery.spec.ts tests (the ones that dispatch a synthetic ChunkLoadError and expect exactly one intercepted reload request) times out at 0 reload requests on a CI shard - specifically shard 1/4, which is where this spec lands (verify with npx playwright test --list --shard=1/4) - including on a clean re-run, while passing reliably on a local machine run in isolation. No error is thrown anywhere in the test itself (page.route() registers fine, page.evaluate() dispatching the synthetic error completes fine) - the reload request simply never arrives. Two independent root causes were found across two rounds of this; both had to be fixed.

Cause #1 (PR #397): useChunkErrorRecovery's guard (chunkErrorRecovery.ts's CHUNK_RELOAD_GUARD_KEY, a real 10s sessionStorage-backed "only one reload per window" debounce, working exactly as designed) gets consumed by a real chunk hiccup before the test's own synthetic dispatch ever runs. CI's playwright.config.ts webServer runs npm run dev (not a prebuilt static export), and Next's dev server compiles pages on demand — the navbar's own "Editor" nav link (visible even while already on /editor) gets prefetched by next/link's default viewport IntersectionObserver behaviour, which can trigger a second on-demand recompile of pages/editor.js mid-test (confirmed via a CI trace: an ~800ms second compile of that exact chunk, network-adjacent in time to the test's own dispatch). A slow/ cold CI runner is more likely to still be mid-churn from that when the test body reaches its own dispatch, and any real transient chunk error during that churn legitimately consumes the guard first. This is a dev-server/test-harness artifact only — the deployed static export has no on-demand compilation or HMR at all, so it can't happen in production. Fixed by clearing the guard immediately before each test's own dispatch - but this alone wasn't sufficient (see cause #2).

Cause #2 (found after #397 merged, PR #395's CI run 30043836352): same symptom recurred on shard 1/4, all 3 attempts including retries. Two independent gaps remained: (a) #397 cleared the guard and dispatched the synthetic error as two separate page.evaluate() round-trips, leaving a real (if narrow) window between them for the same class of dev-server chunk noise cause #1 already identified to land in; (b) loadPageWithDefaultBackend()'s own "Choose Art" click is a raw DOM click - Chromium dispatches it whether or not React has actually hydrated and useChunkErrorRecovery's useEffect has run yet, so a successful click is not proof the recovery listener is live. Reproduced locally: this spec failed intermittently at --workers=4 (parallel Playwright workers contending for CPU with the dev server's own on-demand compilation, closely matching CI's shard-1 composition after PR #395 un-skipped 58 ported parity tests into the same 4 shards - see git log on playwright.config.ts, whose fullyParallel/workers: undefined settings predate this spec and were never the trigger) but passed reliably at --workers=1. A native DOM event dispatched before a listener is attached is simply lost, so this can't be fixed by polling for longer.

Fix: (a) folds the guard-clear and the synthetic dispatch into ONE page.evaluate() call per test (clearGuardAndDispatchChunkError(Error) AsRejection) so nothing on the page's event loop can interleave between them. (b) adds awaitHydrated(), which retries clicking "Choose Art" (mirroring test-utils.ts's own established openAddCardsDropdown() pattern for the identical symptom) until ProjectEditor.tsx's "editor" tab content - specifically CardGrid.tsx's "Your project is empty at the moment." empty-state text - actually becomes visible, which can only happen once Tab.Container's onSelect handler has bound and fired, i.e. once the same hydration/effect-flush pass that mounts useChunkErrorRecovery's own listeners has completed. This needed no extra network dispatch or navigation - two earlier attempts that DID add one (a page.route()-intercepted warm-up reload, and a real uncontrolled warm-up reload followed by re-navigating) were each independently found to destabilise the page even further under this same --workers=4 stress (a SecurityError: ... Access is denied for this document on the aborted-navigation path, and a Navigation ... is interrupted by another navigation on the real-reload path - the self-referential "Editor" nav-link prefetch from cause #1 is a genuinely recurring background navigation under load, not a one-off, and collides with any additional top-level navigation this file issues). The one test that specifically exercises the guard-suppression behaviour (two dispatches in one test) only clears the guard once, before the first dispatch, so the guard is still genuinely exercised for real on the second one. See chunkErrorRecovery.spec.ts's own comments for the full trace-based diagnosis of all three rounds (root-caused against CI runs 30039392833 and 30043836352, both shard 1/4, plus local repro at --workers=4/cold .next cache/CI=true). Verify any future fix here the same way: --shard=1/4 (not the file in isolation) against a cold .next cache, since shard composition (not just raw worker count) drives how much dev-server contention this spec actually sees.

local_calculate_verdicts silently runs with an empty back-face lookup after an image rebuild (Scryfall cache lost)

Symptom: manage.py local_calculate_verdicts --write runs cleanly, finishes COMPLETED, but back-face-only cards (e.g. a card uploaded under a DFC's second-face name) that should resolve via _resolve_candidates_for_card's DFCPair fallback all come back printing_pk=None/no vote. Nothing errors or warns loudly - the only tell (if you go looking) is a per-card logger.warning from _load_back_face_names ("Scryfall bulk-data file not found... back- face lookup returning an empty set"), easy to miss in routine log output. Observed live: deploy-2 (2026-07-23) rebuilt the django/worker images, and the run that followed was silently degraded this way.

Cause: scryfall_cache/default_cards.json (the Scryfall bulk-data cache printing_metadata_import.py/mtg.py's import_canonical_card_data both read/refresh, ~558MB) lived as a plain file inside the django/worker container filesystem with no persistent volume mounted over it (docker/docker-compose.prod.yml had no volumes: entry for either service). Every up --build image rebuild throws away the old container filesystem, taking the cache with it. get_back_face_names/is_back_face (see their own docstrings) correctly treat a missing file as "no back faces known yet" - one warning, frozenset() returned, never a raise - which is the right behaviour for a per-card lookup, but left the whole local_calculate_ verdicts run silently degraded with no failure signal anywhere in its own COMPLETED ledger row or terminal summary.

Fix (issue #402, two parts): (1) scryfall_cache is now a named, persistent Docker volume (docker/docker-compose.prod.yml, mirroring postgres_data/elasticsearch_data's own pattern) mounted on BOTH the django service (manual exec-run commands) and the worker service (the weekly import_canonical_card_data/update_dfcs schedule that runs via its own manage.py qcluster process, per "Startup vs. scheduled catalog sync" above) - a rebuild no longer touches it. Compose changes only take effect at the NEXT deploy (up --build -d), not retroactively on already-running containers. (2) printing_metadata_import.ensure_scryfall_cache_present() is a new fail-loud guard, called at the very start of local_calculate_verdicts's Command.handle() (before any card-by-card work) - it raises a CommandError naming the missing path if the cache file doesn't exist, distinct from get_back_face_names's existing soft "empty set" path, unless --allow-missing-scryfall-cache is passed explicitly. This catches the failure mode structurally even if the volume mount is ever missed again (e.g. a fresh box rebuild that skips the compose file, or a manual docker run bypassing compose entirely).

Scryfall bulk-data refresh aborts on pydantic data.N.download_uri/data.N.size validation errors

Symptom: stream_full_catalog's stage-0 freshness check exits non-zero (EXIT_STAGE_ZERO_FAILED) with a pydantic ValidationError naming data.N.download_uri and data.N.size as missing, for every entry in the /bulk-data response. import_canonical_card_data fails the same way (data.N.download_uri, data.N.size, data.N.content_type, data.N.content_encoding). The on-disk cache is intact and recent, so nothing is wrong with the data - only the refresh path is broken. First seen 2026-07-28.

Cause: Scryfall retired the pre-JSONL bulk format. Per their blog post "Two New Ways to Sync Scryfall Data" (2026-07-01), both formats were offered until 2026-07-20, after which jsonl_download_uri is the only download property and the old files are gone. THREE separate things changed, and fixing only the first leaves a broken importer:

1. Fields: download_uri → jsonl_download_uri, size → compressed_size (and content_type/content_encoding removed outright). Note compressed_size measures the gzipped artefact (~77MB for default_cards) where size measured the uncompressed payload (~620MB) - it still works as a change detector, but it is not the same quantity.
2. Format: the payload is JSONL - one object per line, no wrapping array, no separating commas. Anything doing json.load() over the whole file must read line-by-line.
3. Compression: the file is gzipped on disk (.jsonl.gz, Content-Type: application/gzip), NOT gzip transfer-encoding. The old files carried Content-Encoding: gzip, which an HTTP client decompressed transparently; now you receive a .gz and must inflate it yourself. This is the part most likely to be missed, because the download "succeeds" and lands binary garbage.

Fix: the schema and the index fetch now live once, in cardpicker/integrations/game/scryfall_bulk_data.py, and all three importers resolve a download URL through it by entry type. That module inflates the stream as it arrives, keeps PR #515's atomic temp-file + os.replace swap, and raises BulkDataDownloadError on a non-gzip or truncated body rather than installing a partial catalog. The on-disk cache stays decompressed JSONL at the same scryfall_cache/ default_cards.json path (readers re-read it on every pass; inflating 620MB each time to save disk is the wrong trade), and the shared line reader still tolerates the retired pretty-printed-array shape so a pre-cutover cache already on the persistent volume keeps working until the next refresh replaces it.

Why two of three importers broke and one did not: each importer had its own pydantic model of the same endpoint, so hardening one did nothing for the other two. tests/test_scryfall_bulk_data.py now pins the model against a captured copy of the real /bulk-data response and asserts no importer re-declares a bulk-data field, so the next upstream change surfaces in CI rather than in a production stage-0 abort. An opt-in live check against the real API runs under SCRYFALL_LIVE_CONTRACT=1.

Not adopted: the same blog post announced a /cards/manifest method (a paginated list of which cards exist and when their images last changed). It is an incremental-sync primitive; every consumer here does a whole-catalog pass, and "has anything changed at all" is already answered by the entry's updated_at against the freshness sidecar. Adopting it would be a new incremental-import design, not a fix for this break.

Stage E concurrency cap configured but zero throttled-concurrency-cap outcomes, host load trips anyway

Symptom: settings.STAGE_E_MAX_CONCURRENT_DISPATCHES is set (e.g. the default 2), several django-q2 workers are dispatching Stage E micro-batches concurrently, and yet: (a) DispatchOutcome.status is never "throttled-concurrency-cap" no matter how much concurrent load there is, (b) the worker log carries the literal warning pg_advisory_unlock reported slot N was not held by this connection - possible connection recycling mid-dispatch (cardpicker. stage_e_concurrency's own defensive guard), one occurrence per dispatch, and (c) the host load average envelope bar (operating_envelope, ceiling 7.0) trips anyway, from a burst of CPU-bound Stage C work that the concurrency cap should have prevented from ever starting.

Cause: this exact signature is the 2026-07-25T00:25Z production shakedown (envtrip-20260725T002504-73e1eb6d, {'ceiling': 7.0, 'load_avg': 11.4013671875}) — the cap's advisory lock was held on django.db.connection (Django's shared per-thread connection) instead of a dedicated connection. cardpicker. stage_e_signals's post_save receivers fire during Stage C's persist_evidence, squarely inside the cap's own locked region, and call django_q.tasks.async_task(...), which synchronously calls the installed ORM broker's enqueue — django_q.brokers.orm.ORM. get_connection() calls django.db.close_old_connections() unconditionally whenever not inside an atomic block. With CONN_MAX_AGE unset (this project's DATABASES["default"] has no override, so Django's default 0 applies), that call closes the connection the first time anything asks, not just after some elapsed age. A closed connection auto-releases every Postgres advisory lock its session held, so the cap's own lock died mid-dispatch and every subsequent worker found every slot "free" — the warning above is Postgres reporting that the FINAL unlock call (which runs on a transparently-reconnected, and therefore DIFFERENT, backend session) found nothing to release.

Fix (shipped 2026-07-25, same day): cardpicker/stage_e_concurrency.py now opens a DEDICATED psycopg2 connection (autocommit=True) it alone owns for the lifetime of one try_acquire_dispatch_slot() call — never django.db.connection. See that module's own docstring ("WHAT WENT WRONG IN PRODUCTION" section) and docs/features/stage-e-operations.md's "Concurrency cap" section for the full writeup, and cardpicker/tests/test_stage_e_concurrency.py's TestRegressionDedicatedConnectionSurvivesFollowOnEnqueue for the regression tests that reproduce this exact failure (proven, by hand, to fail against the pre-fix module).

How to confirm it's this (if the warning above recurs after the fix): the warning guard was deliberately KEPT, not removed, specifically so a regression here stays visible — a fresh occurrence means something else has broken the "dedicated connection, never django.db.connection" contract, not that the original bug is back verbatim. Check StageEThrottleCounter.objects.get().count/.last_throttled_at (Django admin, or cardpicker.models.StageEThrottleCounter) — introduced in the same fix as the one durable, queryable signal for whether throttling is happening at all (throttled dispatches write no PilotRunLedger row).

Host load stays high (~4+) with no visible pipeline process, pg_stat_activity shows cardpicker queries hours old

Symptom: uptime/top shows sustained load ~4 or higher with no manage.py/qcluster/dispatch process anywhere in the process list to blame; SELECT * FROM pg_stat_activity WHERE state != 'idle' shows one or more cardpicker_* queries that have been active for hours. Observed 2026-07-25: six pre-#459 SELECT DISTINCT cardpicker_card.id backends kept grinding up to 2h31m after the client containers that issued them were recreated by the 14:03Z deploy.

Cause: Postgres backends survive client death. A backend mid-query never touches its client socket until it has results to send, so it has no way to notice a deploy's container-recreate, a docker exec client being killed, or an OOM kill of the client process — it just keeps computing for a client that no longer exists. Any of those three events mid-long-query reproduces this: silent CPU burn on prod with nothing in the process list to blame, because the actual work is happening inside the Postgres server process, not the (now-dead) Django/worker client.

Fix now: find the orphaned backend(s) in pg_stat_activity (pid, query_start, query) and pg_terminate_backend(<pid>) them via the host's orchestration kill script. Two traps doing this by hand: pkill does not exist in the slim django image (no /bin/pkill binary), so killing the query process from inside that container means a /proc scan-and-kill, not a pkill one-liner; and killing a docker exec client process only kills your shell into the container — it never touches the actual in-container process (or, per this entry's own cause, the Postgres backend it kicked off), so that alone does not stop the orphan grind.

Fix permanently: issue #462 (client_connection_check_interval) — Postgres 14+'s periodic in-query client-socket probe that aborts a query once it notices its client is gone, deliberately not statement_timeout (BULK-mode eligibility scans legitimately run for minutes-hours with a live client and must not be killed on a timer). Parked on the board, not yet shipped.