fix(recap): apply the gate-precision noise floor to the recap's aggregate false-positive rates - #9794
Conversation
…gate false-positive rates `buildMaintainerRecap` re-derived `totals.gateFalsePositiveRate` (and the miner/ human cohort aggregates) with a bare `blocked > 0` guard, while the per-repo path and the Gate-outcomes section both null the rate below MIN_SAMPLE (5) as gate-precision does. So one digest could read "Gate false-positive rate: 100% (2/2)" in its summary and Totals while its own Gate-outcomes section said "n/a (fewer than 5 blocks)" — the maintainer sees "the gate is 100% wrong" and "not enough data" about the same window, from the same totals struct. Export the floor once as `MIN_GATE_PRECISION_SAMPLE` from gate-precision.ts (replacing its private `MIN_SAMPLE`), import it in maintainer-recap-gate-outcomes.ts (deleting the duplicate) and maintainer-recap.ts, and gate the blended rate and each cohort aggregate on `blocked >= MIN_GATE_PRECISION_SAMPLE` against their own denominator. The below-floor case subsumes the old divide-by-zero guard; the rate-line arms and the `number | null` type are unchanged. The 5 now lives in one place, so the next consumer of RecapReport.totals inherits it. Closes JSONbored#9691
|
🚨 Contributor flagged. Click here for more info: Superagent Dashboard |
|
Warning ⏸️ LoopOver review result - manual review recommendedReview updated: 2026-07-29 10:28:38 UTC
Review summary Nits — 1 non-blocking
Decision drivers
Context & advisory signals — never blocks the verdict
Review context
Contributor next steps
Signal definitions
🧪 Chat with LoopOverAsk LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.
Full command reference: https://loopover.ai/docs/loopover-commands 🧪 Experimental — new and may change. Decision record
🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed 💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →. Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.
|
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9794 +/- ##
==========================================
+ Coverage 76.58% 76.65% +0.06%
==========================================
Files 282 285 +3
Lines 59464 59620 +156
Branches 6555 6605 +50
==========================================
+ Hits 45543 45699 +156
Misses 13639 13639
Partials 282 282
Flags with carried forward coverage won't be shown. Click here to find out more.
|
…vider failure (#9806) A review that PARSED cleanly but had every public field withheld by the sanitizer made composeAdvisoryNotes return null, which the orchestration misreports as "AI review is unavailable for this PR head" -- a PROVIDER failure -- and holds the PR for manual review. Because the head is unchanged, every re-run reproduces it: observed live on #9794, which re-ran with diagnostics `claude-code#0:parsed` and was re-held every time, including after explicit maintainer re-ticks. The shape is routine for this project's own scoring/gate code: an honest narrative about it says "score"/"ranking"/"reward" in every sentence, the per-sentence sanitizer drops them all, and a clean review (no blockers, no nits) has nothing left to publish. Publish a fixed, public-safe sentence instead, worded by the review's REAL verdict: "found no blocking issues" only when the model returned no blockers; "raised blocking findings ... read the private review record" when it did -- withheld blockers must never read as a clean result. A truly empty parse (no assessment at all) still returns null, so a genuine provider failure keeps its accurate report. One legacy test pinned the old null; updated with the incident reference.
What & why
services/gate-precision.tsnulls a false-positive rate belowMIN_SAMPLE = 5blocks — "a 1-of-1 false positive is noise".buildMaintainerRecap(src/services/maintainer-recap.ts) folds per-repo precision reports into a cross-repoRecapReportand applied that rule inconsistently inside one function:overall.falsePositiveRatestraight out of the (already-floored)GatePrecisionReport— correct.totals.gateFalsePositiveRateand the miner/human cohort aggregates with a bareblocked > 0guard — unfloored.So one digest could read, over the same window and the same
totals: summary "Gate false-positive rate: 100% (2/2 block(s) later merged)"; Totals "Gate false positives: 2/2 (100%)"; and Gate-outcomes "False-positive rate: n/a (fewer than 5 blocks…)". The maintainer sees "the gate is 100% wrong" and "not enough data to say" simultaneously.MIN_SAMPLE = 5was a private const duplicated in two files, which is what let this third consumer be written without it.The fix
MIN_GATE_PRECISION_SAMPLE = 5fromgate-precision.ts(replacing its privateMIN_SAMPLE).maintainer-recap-gate-outcomes.tsimports it and deletes its duplicate; its rendered "fewer than N blocks" copy uses the imported value.buildMaintainerRecapgates the blended rate and each cohort aggregate onblocked >= MIN_GATE_PRECISION_SAMPLEagainst its own denominator (not the blended one). The below-floor case subsumes the oldblocked === 0divide-by-zero guard; the two rate-line arms and thenumber | nulltype are unchanged.MIN_SAMPLE/5now lives in exactly one place (grep-verifiable) — the next consumer ofRecapReport.totalsinherits it.Tests
test/unit/maintainer-recap.test.ts:blocked 1 / blockedThenMerged 1→totals.gateFalsePositiveRate === null(named regression; fails onmain).blocked 5, fp 1→0.2(floor inclusive at 5).nulleven when the blendedblockedis above it; a cohort at/above its floor still reports (both>=-arms covered).blocked2/3/4) tonull, and aligned therepoInputfixture'scohortReportto gate-precision's real>= 5floor.test/unit/maintainer-recap-format.test.ts:- Gate false positives: 2/2 (n/a)and the Gate-outcomes "fewer than 5 blocks" line, with no percentage anywhere (Deliverable 5).Validation
npm run typecheckgreen; all four affected suites (58 tests) green.git diff --check <base> HEADclean; no route/schema/migration change (RecapReporttype unchanged).Closes #9691