perf(task-board): cache the assembled PR card, and never wait on GitHub for what the DB has

[pedrofrxncx]

Rebased on origin/main.

The measurement

pr_read_cache_fetches_total in prod, last hour:

outcome	count
miss (blocks on GitHub)	922
stale (served instantly)	1273
hit	81
error	26

A miss blocks on four GitHub round-trips. That is the ~2s on opening a task — including on a page refresh well inside the 30-minute window, where it should have been a hit.

Fix 1 — cache the card, not the raw reads

The read cache stores raw GitHub payloads, and get_comments on a busy PR runs past the 512KB maxValueSize. The put is rejected, the rejection was swallowed by a bare catch {}, and that PR then misses on every read forever — silently, and indistinguishably from a cold cache. The busier the PR, the more certain it never caches.

A TaskBoardItemPr[] is a few hundred bytes, so it always stores, and one KV get replaces the four-to-six this made per PR. maxStaleMs is a day, not half an hour: past it the card blocks again, and a card rendered once should never make someone wait on GitHub twice. Stale is still refreshed on every poll, so this trades nothing but the worst case.

Fix 2 — never block for what the database already has

The repo, the number and the link are in task_board_item_prs the whole time; only the title, checks and preview need GitHub. fetchOrPlaceholder returns the DB row immediately on a cold card and runs the GitHub read in the background, so the enrichment lands in KV for the next poll instead of holding the response.

Two details that matter:

• The reconciles are skipped on the placeholder. All-null means "we have not asked yet", not "open, unmerged, no checks" — prReadyForReview would read it as a candidate and hand the card to the reviewer on the strength of a database row. They run on the next poll, against real data.
• A stale card is still served as live, not replaced by the placeholder. Falling back there would blank a fully-rendered card back to a bare link.

While any card is unenriched the dialog polls every 2s instead of 60s, so the fill-in is a moment rather than a minute.

Make the failure visible

Both caches now run on one generalized JetStreamKVPrCache (namespace + key, separate buckets, a cache label on the metric), and a rejected put increments store_rejected instead of vanishing. That counter is what would have surfaced fix 1 in an hour rather than a week. placeholder is a new outcome too, so the cold-card rate is visible.

The read cache stays — it still serves the archive/review sweeps, which read many distinct PRs and benefit from payload-level entries.

Testing

pr-cache.test.ts (renamed from pr-read-cache.test.ts; the file no longer only caches reads) — 9 tests:

• the original 3 SWR cases (serve-then-revalidate, keep serving on a failed refresh, invalidate)
• card-cache config invariants: longer max-stale than the read cache, sub-poll revalidate, distinct buckets
• a bucket that rejects every put, pinning the "answers correctly but never caches" mode
• fetchOrPlaceholder: returns the placeholder without waiting then serves the real value; a stale value stays live; past max-stale it falls back rather than blocking

bun run --cwd=apps/api check, bun run lint, knip clean. Pre-existing on main and unrelated: apps/web check has a duplicate-prosemirror-version error, and the 10 *.integration.test.ts in task-board/ need Postgres.

What to watch after deploy

sum by (cache, outcome) (increase(pr_read_cache_fetches_total[1h])). Expect cache="cards" to be overwhelmingly hit/stale, with placeholder only on genuinely cold cards. A non-zero store_rejected on cards means a check-run summary is blowing the cap and should be truncated before storing.

Follow-up, not in scope

The card cache is storage: Memory, so a NATS restart empties it and every card goes through the placeholder path once. Persisting the last-known card on the task_board_item_prs row would survive that — a migration plus a write path.