feat(review): structured verifiable absence-claim schema — hallucinated absence can no longer block (#8833)

[JSONbored]

What

The flagship remaining piece of #8833, stacked on #9592 (this branch contains its commit until it merges; the diff collapses to just this change after).

The rubric's DIFF SCOPE / TRACE-BEFORE-ASSERTING-ABSENCE rules have always requested that an absence blocker point at the visible line that breaks — the confirmed hallucination pattern ("missing import/guard/handler" the model merely could not see) walked straight past prose. This makes the discipline checkable, then enforces it:

• Object-form blockers: entries may be {"claim", "kind", "evidence"}; the object form is REQUIRED for the absence family (missing_symbol | missing_import | missing_guard | missing_handling | missing_registration — a closed set, so a novel kind cannot smuggle a claim past verification). parseModelReview normalizes every usable entry into the existing blockers: string[] surface — nothing downstream changes shape — and collects absence claims into ModelReview.absenceClaims.
• Mechanical verification: demoteUnverifiedAbsenceBlockers checks each claim's quoted evidence verbatim against the exact user prompt the call sent (leading diff +/- marker stripped; sub-8-char quotes never verify vacuously). Unverifiable ⇒ demoted to the very verification nit the rubric prescribes ("Verify: …"). Verified ⇒ the blocker stands untouched — the check can only remove claims the model provably could not ground, never real judgment.
• Enforced on both the Workers and BYOK parse paths, logged as ai_review_unverified_absence_demoted (a rising rate is a prompt-regression signal, same as the sibling demotions).
• REVIEW_PROMPT_VERSION → v3; the ai(cache): review fingerprint omits prompt and verdict-logic version — superseded verdicts replay forever (11 unbumped changes) #9477 cache fingerprint makes the drift an automatic cache miss and counterfactual: advisory check when reviewer prompt versions change #8222's replay keys on it.

Why this closes #8833

With this, every objectively-checkable criterion in the inventory (posted on the issue, 2026-07-26) has a deterministic owner the model cannot override: CI state (#8845), test-evidence existence (#9256), evidence-absence under truncation (#8980), size + staleness/conflicts and the explicit "unknown" abstention (#9592), scanner-confirmed secrets/lockfile facts (rule-produced findings), and now the absence-claim family — machine-checked, not pleaded. The issue's per-dimension isolated-judge-call architecture is deliberately not adopted: it multiplies per-review AI spend several-fold to re-derive what the demotion family already guarantees by construction, and the confidence/abstention machinery (#8834's both halves + the unknown→hold route) now covers exactly the subjective residual. The remaining REES-side item (structured CVE facts from the enrichment service) needs the REES response schema and is tracked in the enrichment tree, not this gate issue.

Tests

• Parser: mixed string/object forms; whitespace/claimless/non-string entries dropped fail-safe; non-string evidence normalizes to "" (never verifies); novel kinds never collect evidence; the 6-blocker cap drops overflow claims too (no orphaned verification entries).
• Verification: verified-vs-hallucinated side by side (verified stands, hallucinated demotes to Verify: nit); diff-marker stripping; vacuous-quote floor; zero-claim same-object return.
• REGRESSION (both paths): runWorkersOpinion verifies against the actual prompt it sent; the BYOK path demotes an ungrounded object-form claim and fires the event.
• Prompt-family suites (integrity, cache fingerprint, counterfactual replay) green on v3. 100% of changed lines and branches covered.

Closes #8833

[loopover-orb]

Tip

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-28 21:36:13 UTC}

3 files · 1 AI reviewer · no blockers · readiness 98/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
This PR adds an object-form blocker schema `{claim, kind, evidence}` for absence-family claims (missing_symbol/import/guard/handling/registration) and mechanically verifies the quoted evidence against the actual user prompt sent to the model, demoting unverifiable claims to nits via the new `demoteUnverifiedAbsenceBlockers`. The wiring is threaded correctly through both the Workers (`runWorkersOpinion`) and BYOK provider (`runProviderReview`) paths using the same `user` prompt text that was actually sent, `parseModelReview` normalizes both string and object blocker forms into the existing `blockers: string[]` surface so nothing downstream changes shape, and the closed `ABSENCE_CLAIM_KINDS` set prevents a novel kind from smuggling an unverified claim past the check. `REVIEW_PROMPT_VERSION` is correctly bumped to v3 with the matching replay test updated, and the accompanying tests (object-form parsing, 6-blocker cap dropping orphaned absence claims, marker-stripped/vacuous-quote edge cases, end-to-end demotion on both paths) exercise the real logic rather than fabricated scenarios.

Nits — 4 non-blocking

• src/services/ai-review.ts: `toBlockers`'s `absenceClaims.filter((a) => keptSet.has(a.claim))` matches by claim TEXT, so two entries (a plain string and an object) that happen to share identical claim text could cross-attach evidence — worth keying by array index/identity instead of string equality if that's a real concern for you.
• The external brief's console.warn flags at ai-review.ts:1763/2168 aren't debug leftovers — they follow the exact same structured `console.warn(JSON.stringify(...))` pattern every sibling demotion (`ai_review_ci_claim_demoted`, `ai_review_size_claim_demoted`, etc.) already uses.
• The `6` cap in `toBlockers` (ai-review.ts:1182) mirrors the existing `toList` blocker cap rather than introducing a new value — consider a shared named constant if both are meant to stay in sync.
• Consider whether the evidence-matching should also tolerate a leading `-` diff marker paired with trailing whitespace variations beyond the single optional space already handled, in case models quote removed lines with different spacing.

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8833
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 14 registered-repo PR(s), 13 merged, 276 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 14 PR(s), 276 issue(s).
Improvement	✅ Minor	risk: clean · value: minor · LLM: significant

Linked issue satisfaction

Partially addressed
This PR delivers one concrete deterministic-check extraction (mechanically verifying absence-claim evidence against the prompt, with solid both-arm tests and a regression test) that fits the issue's pattern of moving objectively-checkable criteria out of LLM judgment, but the issue's broader asks—an explicit inventory table in the PR description, per-dimension isolated judge calls with an 'unknown

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is registered but has no active allocation in the current snapshot.
• Public profile languages: Python, TypeScript, Ruby, Go, MDX, Shell, Solidity, JavaScript
• Official Gittensor activity: 14 PR(s), 276 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
• Then work through the remaining 1 step in the Signals table above.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[github-actions]

Logic backtest

Replayed 0 historical case(s) for linked_issue_scope_mismatch through the base (b19111d) and head (a394dda) versions of its detection logic (corpus checksum 4f53cda18c2b).

Backtest comparison: linked_issue_scope_mismatch

Verdict: unchanged — no comparable axis moved.

Advisory only — this check never blocks merge (#8105).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.04%. Comparing base (b19111d) to head (a394dda).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9593      +/-   ##
==========================================
- Coverage   89.84%   89.04%   -0.81%     
==========================================
  Files         880      880              
  Lines      111137   111175      +38     
  Branches    26457    26473      +16     
==========================================
- Hits        99853    98994     -859     
- Misses       9992    11155    +1163     
+ Partials     1292     1026     -266     

Flag	Coverage Δ
backend	94.16% <100.00%> (-1.46%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
src/services/ai-review.ts	97.05% <100.00%> (+0.13%)	⬆️

... and 3 files with indirect coverage changes

[loopover-orb]

LoopOver approves — the gate is satisfied and CI is green.