Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

review-discipline

Code review that has to prove its findings — 8 enforceable rules, benchmarked. Every published review skill is a checklist, and checklists create tunnel vision: in our public bench, a checklist-guided review missed a live P1 (hardcoded secret) that the unguided baseline caught, because the list had no line for it. This skill inverts the shape — open hunt before any checklist, a demonstrated failure path for every finding (or an explicit [suspicion] downgrade), and a self-refutation pass before anything gets reported.

Built and bench-tested by SkillProof, the tested Claude skills directory.

What it does differently

We surveyed 321 crawled review skills plus the standalone field (including Anthropic's own code-review plugin) before writing this one. Three mechanics showed up in none of them as enforceable single-agent rules — they are this skill's core:

  1. No demonstrated failure path, no finding. Every finding must name a concrete input or state that reaches the flagged code and produces a wrong outcome. Can't construct it? It ships as [suspicion], ranked below every proven finding — never as fact. Anthropic's plugin filters false positives with parallel scorer agents voting a confidence number; we filter with a demonstrated failure trace, in one agent, in one installable file.
  2. Two passes, open hunt first. Read the change as an attacker with no category list in hand; only then sweep the standard checklist (secrets, injection, boundaries, concurrency…). Every competitor is the checklist. Our bench evidence for why the order matters is the missed live P1 above.
  3. Kill your own finding before reporting it. One honest refutation attempt per finding — upstream guards, intentional behavior, existing test coverage, reachability — and the report says what was checked. Findings that die, die silently.

Plus the rest: P0–P3 severity with definitions (not vibes), diff-scope discipline (review the change, not the codebase — with a carve-out for pre-existing live secrets), missing-test callouts tied to the changed behavior, zero praise filler, and a fixed output contract: severity, file:line, failure path, what was checked, suggested fix.

Install

git clone https://github.com/Skillproofdev/review-discipline ~/.claude/skills/review-discipline
# restart Claude Code — triggers on "review this PR/diff", "find bugs in", pre-merge checks

One command — the repo IS the skill.

Benchmark (measured, not estimated)

Seeded-bugs protocol: real code samples (~200–400 lines each) seeded with documented bugs of known severity — logic, security, edge-case, concurrency — with ground truth (location, failure path, detection criteria) committed before any run. Real intact code stays in each sample to measure false positives. Same prompts, same model; the only difference is whether the agent reads this SKILL.md first. Full methodology: skillproof.dev/methodology.

4 samples (TypeScript / Python / JS), 21 seeded bugs, 12 intentionally-healthy false-positive traps. Base agent vs. the same agent reading this SKILL.md first; identical prompts, same model.

Metric Base (no skill) With skill
Seeded bugs caught (of 21) 17.0 (81%) 18.5 (88%)
P0 catch (exploitable / data-loss) 3/3 3/3
P1 catch (wrong behavior, realistic path) 7/7 7/7
P2 catch (edge paths) 6/7 7/7
Hardcoded-secret probe caught caught
False-positive rate 5.6% (1 FP) 0%
Findings with a demonstrated failure path ~63% ~100%
Undemonstrable claims flagged [suspicion] 0 5
Praise / filler lines present none

Honest read (negative results included): every P0 and P1 merge-blocker — including the hardcoded-secret probe — is caught 100% by both arms. On top of that, the skill catches more total bugs than the unguided baseline (18.5 vs 17 of 21), driven by a clean 7/7 on medium edge-path bugs where breadth-before-depth sweeps the rounding and empty-input cases a baseline skips — while holding zero false positives (baseline had one), a demonstrated failure path on ~100% of findings vs. ~63%, and honest [suspicion] downgrades instead of asserting hunches as fact. One caveat we don't hide: an earlier version of this skill (v1) uniquely caught one subtle deep-nesting bug that the current breadth-first version walks past — coverage is broad but not a strict superset of every prior run. Full 3-way adjudication (base vs v1 vs v2): bench/results/verdict.md.

When it triggers

"Review this code/PR/diff", "check this before merge", "find bugs in", pre-merge checks, "any problems with this change". Explicitly excluded: writing new features, style-only linting, formatting disputes, prose review.

Why trust this

We test other people's skills for a living (public methodology). Our own skills get the same treatment — measured benchmarks with pre-registered ground truth, negative results included. The checklist failure that motivated this skill was found in our own public bench of a skill we had rated pass.

More skills from us: skillproof-skills index · Free tools for skill authors: SKILL.md validator, token calculator, Rules⇄SKILL.md converter.

License

MIT — use it, fork it, ship it.

About

Code-review skill for Claude that proves every finding (demonstrated failure path or [suspicion]) and hunts breadth-before-depth. Seeded-bug benchmark: 18.5/21 caught, 0 false positives.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages