docs(specs): guardian skeptic — per-finding judgement + impact scoring (#246)

[zaebee]

Refs #246. Design only — no implementation.

Why now

The recall-lean finder (#249) delegates precision to the skeptic. The skeptic is not delivering, and the live run on #263 today shows why in one picture: 8 findings, all confirmed, all at one severity. One was a real defect (Path.is_file() raises on EACCES/ENAMETOOLONG — accepted and fixed). Two were factually wrong (for size: int in range(...) is not valid Python; as_posix() would have broken the Windows-path case the line exists for). Nothing in the output separated them.

Two causes, only one of which is the model:

1. Mechanism — one call carries the whole findings list, verdicts mapped back by finding_index. Everything competes for one attention window, and index mapping has its own failure modes.
2. Contract — Literal["confirmed","refuted","uncertain"] expresses "this is false" but not "this is true and worthless", and cannot be sorted or thresholded.

Swapping the skeptic model under an unchanged mechanism buys the same flat output at higher cost. So #246 becomes mechanism-first; the model matrix runs on top as phase 2.

What the design proposes

• Two axes. verdict (truth, unchanged semantics — refuted drops) + impact_score: int 0-10 (importance — below threshold hides, never drops, and stays in the JSONL so the benchmark can still see it).
• One call per finding, carrying only that finding's own hunks from diff_index, bounded concurrency (default 3, mistral's per-minute cap). finding_index disappears along with its failure modes.
• Partial degradation. skeptic_status gains partial with judged/total; one failed call costs one verdict instead of the whole pass.
• A previously-reverted lever becomes usable. Few-shot false-positive examples swung mistral into refute-all when there was one axis, because refutation was the only way to express "bad finding". On the importance axis they cannot reach the refutation logic. The truth-axis prompt is reused verbatim — it was bought by reverting a refute-by-default version that killed 7/7 findings including 2 GT matches.
• "If ruff/mypy would catch it, impact ≤ 2" — Gemini-review advice placed on the axis where it is safe. In the finder prompt the same instruction re-introduces the precision bias that caused recall 0/5.
• Benchmark by A/B over frozen finder output. Both arms judge the same recorded findings, same model, batch vs per-finding — so movement is attributable to the mechanism. Runs at threshold 0 and records every score, so the recall/noise-vs-threshold curve is computed offline from one run instead of one paid run per candidate value.
• Falsifiable gate: noise down and zero lost GT versus the batch arm, plus median(score | GT) − median(score | non-GT) ≥ 3 pooled across PRs. A flat distribution fails even if noise happens to drop — that is exactly the fix(context): resolve source_root against stored paths, first match wins (#228) #263 pathology.

Amends 2026-06-10-guardian-sprint-design.md §5.2 ("one call, not N"), which predates the recall-lean finder: with a ≤5-finding gated finder one call was sufficient; with 10–26 it is the bottleneck.

Phase 2 (paid model matrix) does not start without an explicit budget go — the prior gemini run hit the 30 PLN cap.

🤖 Generated with Claude Code

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud