You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
getCachedAiReview replays a cacheable=1 row unboundedly (there is no maxAgeMs), and the cache fingerprint does not include the reviewer prompt, the verdict-combination logic version, or several per-repo flags that change what the model is asked. So a verdict produced under superseded policy is replayed verbatim — including a severity: "critical"ai_consensus_defect — and the fixed code never runs for that head.
src/db/repositories.ts:5331-5366, :5385-5415 — getCachedAiReview and getCachedAiReviewAcrossHeads reuse durable rows with no age bound; stored findings are replayed as-is.
REVIEW_PROMPT_VERSION (src/services/ai-review.ts:1290) is still "review-prompt-v1" despite its own doc instructing a bump on any verdict-surface change — which additionally means counterfactual: advisory check when reviewer prompt versions change #8222's counterfactual replay gate silently no-ops.
systemPromptDigestis computed (:2720) but reaches only the decision record, never the cache key.
Also absent from the fingerprint: findingCategories and improvementSignal (both append system-prompt suffixes — :1268, :1278), screenshotEvidenceSummary (appended to the system prompt, :1320), AI_MAX_OUTPUT_TOKENS, and the operator plan's onMerge (reviewerPlan hashes only combine and reviewers).
Concrete failure
A PR is closed or held under the pre-#9074 "false consensus" logic. #9074 lands, requiring genuine agreement. The PR is re-gated at the same head — identical fingerprint, cache hit — so the old critical finding and its close rationale are replayed and the corrected logic never executes. The fix is invisible for every already-reviewed head.
Requirements
Anything that changes what the model is asked, or how its answer is judged, must change the cache key.
Correctness must not depend on a human remembering to bump a constant — that mechanism has already failed 11 times.
Deliverables
Fold REVIEW_PROMPT_VERSIONandsha256(buildCanonicalJudgePrompt()) into the cache payload. Both are pure and available before the call, so prompt drift becomes an automatic miss rather than a manual bump.
Add the missing per-repo/plan inputs: findingCategories, improvementSignal, screenshotEvidenceSummary, AI_MAX_OUTPUT_TOKENS, onMerge.
Bump AI_REVIEW_CACHE_INPUT_VERSION once as part of this change, to invalidate rows written under the incomplete fingerprint.
Consider a maxAgeMs backstop on durable reuse, so a fingerprint gap can never mean "forever".
Add a drift check: fail CI when a file that contributes to the prompt changes without the digest mechanism covering it — or, better, make the digest structural so the check is unnecessary.
Tests (must fail against current main)
Same head, prompt text changed ⇒ cache miss.
Same head, findingCategories / improvementSignal / onMerge toggled ⇒ cache miss for each.
Same head, nothing changed ⇒ cache hit (no regression in hit rate for the genuine case).
A row written under v6 is not served after the bump.
Expected outcome
A policy or prompt fix takes effect on the next pass for every PR, instead of being masked by a cache row that predates it.
Summary
getCachedAiReviewreplays acacheable=1row unboundedly (there is nomaxAgeMs), and the cache fingerprint does not include the reviewer prompt, the verdict-combination logic version, or several per-repo flags that change what the model is asked. So a verdict produced under superseded policy is replayed verbatim — including aseverity: "critical"ai_consensus_defect— and the fixed code never runs for that head.Mechanism (verified at HEAD, 776c414)
src/db/repositories.ts:5331-5366,:5385-5415—getCachedAiReviewandgetCachedAiReviewAcrossHeadsreuse durable rows with no age bound; storedfindingsare replayed as-is.AI_REVIEW_CACHE_INPUT_VERSIONisv6(src/review/ai-review-cache-input.ts:17), last bumped for AI CLI timeout knobs are env-var-only, unlike the sibling model/effort knobs (config-as-code gap) #8364. Since then fix(ai-review): the incoherent-diff bail misfires on title-vs-diff scope mismatch, turning reviewable PRs into inconclusive holds #8789, fix(ai-review): inconclusive-finding copy claims 'dual-model' in single+fallback mode, and claude_code auth errors lack structural circuit-breaker classification #8791, gate: move every objectively-checkable criterion from LLM judgment to deterministic checks #8833, fix(orb): enforce the model/fact boundary — CI claims demoted at parse, silence is not certainty #8845, Review-input truncation causes false 'missing screenshot evidence' closes #8961, orb(security): AI reviewer prompt has no instruction-hierarchy defense — untrusted PR text is concatenated raw #9035, orb(ai-review): 'AI reviewers agree' is never verified — consensus fires whenever BOTH have any blocker, not the same one (427 closes) #9074, orb(ai-review): a model that declared it could NOT review the diff can still emit blockers that close the PR #9087, fix(ai-review): verify reviewers actually agree; strip blocker authority from a bailed review #9114, fix(ai-review): fence untrusted PR content, stop shifting inline anchors, and stop judging a fix you cannot see #9145, fix(review): filter the AI narrative assessment per sentence, not all-or-nothing #9445 all changed prompt text or verdict logic without a bump.REVIEW_PROMPT_VERSION(src/services/ai-review.ts:1290) is still"review-prompt-v1"despite its own doc instructing a bump on any verdict-surface change — which additionally means counterfactual: advisory check when reviewer prompt versions change #8222's counterfactual replay gate silently no-ops.systemPromptDigestis computed (:2720) but reaches only the decision record, never the cache key.Also absent from the fingerprint:
findingCategoriesandimprovementSignal(both append system-prompt suffixes —:1268,:1278),screenshotEvidenceSummary(appended to the system prompt,:1320),AI_MAX_OUTPUT_TOKENS, and the operator plan'sonMerge(reviewerPlanhashes onlycombineandreviewers).Concrete failure
A PR is closed or held under the pre-#9074 "false consensus" logic. #9074 lands, requiring genuine agreement. The PR is re-gated at the same head — identical fingerprint, cache hit — so the old critical finding and its close rationale are replayed and the corrected logic never executes. The fix is invisible for every already-reviewed head.
Requirements
Deliverables
REVIEW_PROMPT_VERSIONandsha256(buildCanonicalJudgePrompt())into the cache payload. Both are pure and available before the call, so prompt drift becomes an automatic miss rather than a manual bump.findingCategories,improvementSignal,screenshotEvidenceSummary,AI_MAX_OUTPUT_TOKENS,onMerge.AI_REVIEW_CACHE_INPUT_VERSIONonce as part of this change, to invalidate rows written under the incomplete fingerprint.maxAgeMsbackstop on durable reuse, so a fingerprint gap can never mean "forever".Tests (must fail against current main)
findingCategories/improvementSignal/onMergetoggled ⇒ cache miss for each.v6is not served after the bump.Expected outcome
A policy or prompt fix takes effect on the next pass for every PR, instead of being masked by a cache row that predates it.