feat(review): ban size and stale-base adjudication at parse time, add the explicit unknown abstention (#8833) - #9592
Conversation
… the explicit unknown abstention (#8833) Two more objectively-checkable facts move behind the deterministic boundary, enforced the same way the CI-state ban (#8845) and the test-evidence ban already are -- demoted at parse time on BOTH the Workers and BYOK paths, so no route escapes: - SIZE_CLAIM_PATTERN / demoteSizeClaimBlockers: whole-PR size claims ('too large', 'should be split') are owned by sizeGateMode -> oversized_pr. Anchored on a PR/changeset noun so code-magnitude judgments ('this buffer is too large') never match. - STALE_BASE_CLAIM_PATTERN / demoteStaleBaseClaimBlockers: staleness / rebase / merge-conflict claims are owned by stale_base_ref and GitHub's own mergeable_state -- none of them is even visible in a diff. Claim-shaped so code ABOUT rebasing/conflicts never matches. The rubric now also OFFERS an explicit abstention: a model that genuinely cannot judge its blockers sets confidence to the string "unknown", which parseReviewConfidence maps to CONFIDENCE_WHEN_UNSTATED (0.5) -- under every sane close floor, so an honest abstain routes to the low-confidence hold disposition instead of a guessed verdict. REVIEW_PROMPT_VERSION bumps to v2 (both edits shape the judge's verdict surface), which the #9477 cache fingerprint and the #8222 counterfactual replay both key on. Part of #8833; the structured verifiable-blocker schema lands separately.
|
Warning ⏸️ LoopOver review result - manual review recommendedReview updated: 2026-07-28 21:13:16 UTC
Review summary Nits — 5 non-blocking
Concerns raised — review before merging
📋 Copy for AI agents — paste into your coding agentDecision drivers
Context & advisory signals — never blocks the verdict
Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Decision record
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Logic backtestReplayed 0 historical case(s) for Backtest comparison:
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9592 +/- ##
==========================================
- Coverage 89.84% 89.03% -0.81%
==========================================
Files 879 879
Lines 111071 111095 +24
Branches 26439 26449 +10
==========================================
- Hits 99787 98914 -873
- Misses 9992 11155 +1163
+ Partials 1292 1026 -266
Flags with carried forward coverage won't be shown. Click here to find out more.
|
What
Two more objectively-checkable facts move behind the deterministic boundary, enforced exactly the way the CI-state ban (#8845) and the test-evidence ban already are — demoted at parse time on BOTH the Workers and BYOK paths, so no parse route escapes:
sizeGateMode→oversized_prSIZE_CLAIM_PATTERN→demoteSizeClaimBlockers, unconditionalfetchBaseAheadBy→stale_base_ref; GitHub's ownmergeable_stateSTALE_BASE_CLAIM_PATTERN→demoteStaleBaseClaimBlockers, unconditionalBoth patterns are deliberately claim-shaped: a code-magnitude judgment ("this buffer is too large"), "this function should be split into smaller helpers", or code ABOUT rebasing/conflict-resolution never match — the ban covers assertions about the SUBMISSION and repository state, which are not even visible in a diff.
Explicit abstention (the issue's "unknown → hold" requirement): the rubric now offers
"confidence": "unknown"for a model that genuinely cannot judge its blockers.parseReviewConfidencemaps it toCONFIDENCE_WHEN_UNSTATED(0.5), under every sane close floor, so an honest abstain routes to the low-confidence hold disposition (a human decides) instead of a guessed verdict.REVIEW_PROMPT_VERSIONbumps to v2 — both edits shape the judge's verdict surface, which the #9477 cache fingerprint (drift = automatic cache miss) and the #8222 counterfactual replay key on.Tests
runWorkersOpiniondemotes both claim kinds and logsai_review_size_claim_demoted/ai_review_stale_base_claim_demoted; the BYOK provider path does the same — the LLM path can no longer flip either fact.parseReviewConfidence("unknown")lands under the 0.93 default close floor.Part of #8833 — the structured verifiable-blocker schema (
{claim_kind, path, line, evidence_ref}) lands as its own PR next.