Repository navigation
v2.0.1 — accuracy-per-dollar
PR Sentinel v2: review accuracy as a systems problem, not a model-size problem. The same cheap model, wrapped in a system that samples, anchors, and verifies, beats its own naive single pass — measured, with published false-positive rates.
The leaderboard (same deepseek-v4-flash, 51 fixture-runs)
| Config | Caught | Clean-fixture false positives | Cost/review |
|---|---|---|---|
| Naive single pass | 47/51 (92%) | 2 | ~$0.002 |
| v2 (ensemble + verifier) | 49/51 (96%) | 0 | ~$0.005 |
What is new in v2
- Line-numbered diffs + evidence anchoring — every finding must quote a line that literally exists in the diff, or it is dropped before posting. Hallucinations become structurally impossible.
- Self-consistency ensemble — each analyst reviews 3× and findings are majority-voted (DeepSeek caching keeps this ~1.3x cost, not 3x).
- Verifier pass — a dedicated agent adjudicates every finding against the code before the reviewer writes prose.
- Inline review comments — findings post anchored to diff lines; the rest stay in one sticky summary.
- On-demand commands —
@pr-sentinel review | ask | describe, gated to repo owners/members/collaborators. - Native Anthropic provider, two-tier model routing, JSON-mode with fallback, pooled connections, cache-aware cost reporting.
- 17-fixture / 5-language eval suite (Python/JS/TS/Go/Java) with hard negatives and two prompt-injection vectors.
163 tests, all LLM/GitHub mocked. Live-verified end-to-end: inline comments, cross-agent attribution, and the ask command answering a grounded question about a real PR.
Install: uses: moazmo/pr-sentinel@v2. See the README for the workflow and config.