Confidence gates, evidence grading, and adversarial verification for AI coding agents.
Don't trust AI output. Prove it first.
Quick Start • The Problem • Core Workflows • Comparison • 中文文档
AI coding agents are confidently wrong.
They don't just make mistakes — they make mistakes while sounding absolutely certain. Karpathy nailed the surface symptoms: wrong assumptions, overengineering, drive-by refactoring. But those are the easy failures.
The hard failures are:
- 🔴 Confident misdiagnosis — "The root cause is X" (it wasn't)
- 🔴 Unverified fixes — "Fixed!" (it compiled, but the bug is still there)
- 🔴 Invisible regressions — Fixing A quietly breaks B, C, and D
- 🔴 Assumption laundering — Guesses presented as confirmed facts
- 🔴 Fixation loops — Same wrong approach attempted 5 times
These failures cost hours, not minutes. And no amount of "think before coding" rules will catch them.
Proof-First Development — a verification methodology for AI coding agents.
Instead of telling AI how to code, Proof-First tells AI how to verify its own reasoning.
Three core mechanisms that no other system provides:
Before any code change, the agent must pass a three-round confidence check:
Round 1 — Positive proof: What evidence supports this conclusion?
Round 2 — Adversarial: What evidence would DISPROVE it?
Round 3 — Blast radius: What else could break?
If confidence < 95%, the agent stops and asks instead of guessing.
Every claim is tagged with its evidence level:
| Grade | Meaning | Action |
|---|---|---|
🟢 confirmed |
Verified in code/tests/runtime | Safe to act on |
🟡 inferred |
Logical reasoning, not yet verified | Proceed with caution |
🔴 hypothesis |
Unverified assumption | Must verify before acting |
This prevents the #1 AI failure mode: presenting guesses as facts.
When the same fix fails twice:
STOP → Discard current hypothesis → Generate 2+ new candidates
→ Force adversarial challenge on each → Pick the one with strongest evidence
No more fixation loops. No more "let me try the same thing again but slightly different."
curl -o CLAUDE.md https://raw.githubusercontent.com/user/proof-first/main/adapters/claude-code/CLAUDE.mdcp -r proof-first/workflows/ .windsurf/workflows/cp -r proof-first/adapters/cursor/.cursor/ .cursor/Start with the 3 core files. Add more as needed:
# Minimum viable setup (recommended for first-time users)
cp workflows/confidence-gate.md your-project/
cp workflows/coding-baseline.md your-project/
cp workflows/fix-rethink.md your-project/| Workflow | What it does |
|---|---|
| coding-baseline | 4 non-negotiable rules: ask first, minimal code, surgical changes, define success criteria |
| Workflow | What it does |
|---|---|
| confidence-gate | Three-round confidence check before any code change. Blocks action below threshold |
| deep-analysis | Multi-hypothesis root cause analysis with evidence grading |
| self-audit | Forced self-contradiction before outputting any conclusion |
| fix-rethink | After 2 failed fixes, forces a complete perspective switch |
| Workflow | What it does |
|---|---|
| step-exec | One change at a time. Verify before proceeding. Never stack unverified changes |
| batch-exec | Multi-task execution with per-task verification and global regression check |
| final-review | Pre-delivery checklist: functionality, regression, multi-perspective review |
| Workflow | What it does |
|---|---|
| checkpoint | Save/restore work state across sessions. Never lose context |
| learn-log | Cross-session knowledge accumulation. Same mistake never happens twice |
| retro | Data-driven engineering retrospective with git metrics |
| Workflow | What it does |
|---|---|
| architecture-guard | Layer compliance, SOLID principles, Architecture Decision Records |
| security-audit | Credential scanning, network security, data protection, permission review |
| feature-360 | Feature review from 3 angles: user needs, code logic, industry benchmarks |
| market-gap | Competitive analysis and gap identification |
| report-audit | Verify accuracy of generated reports against actual code |
| Extension | What it does |
|---|---|
| apple-platform | iOS/macOS-specific: Swift patterns, Keychain audit, Widget compatibility, XCTest integration |
Workflows are not isolated — they route to each other:
User Request
│
▼
coding-baseline ← always active
│
▼
deep-analysis ──→ confidence-gate ──→ step-exec ──→ final-review
│ │ │ │
│ (< 95%?) (failed?) (issues?)
│ │ │ │
│ ▼ ▼ ▼
│ STOP & ASK fix-rethink step-exec
│ │
│ (still failing?)
│ │
│ ▼
│ ESCALATE
│
└──→ learn-log (after completion)
└──→ checkpoint (end of session)
| Capability | proof-first | Karpathy Skills | Everything Claude Code | Agent Skills |
|---|---|---|---|---|
| Confidence gate (numeric threshold) | ✅ | ❌ | ❌ | ❌ |
| Evidence grading (🟢🟡🔴) | ✅ | ❌ | ❌ | ❌ |
| Fix-rethink (forced perspective switch) | ✅ | ❌ | ❌ | ❌ |
| Multi-round adversarial verification | ✅ | ❌ | ❌ | ❌ |
| Cross-workflow routing | ✅ | ❌ | ❌ | ❌ |
| Cross-session memory (checkpoint + learn-log) | ✅ | ❌ | partial | ❌ |
| Escalation rules (auto-stop after N failures) | ✅ | ❌ | ❌ | ❌ |
| Anti-rationalization tables | ❌ | ❌ | ❌ | ✅ |
| Multi-language coding rules | ❌ | ❌ | ✅ | ❌ |
| Plugin/marketplace system | ❌ | ❌ | ✅ | ❌ |
| Behavioral coding rules | ✅ | ✅ | ✅ | ✅ |
| Platform-specific audits (Apple) | ✅ | ❌ | partial | ❌ |
In short: Others tell AI how to code. Proof-First tells AI how to verify its own reasoning.
"The problem with AI coding is not that it can't code. It's that it can't tell you when it doesn't know."
Proof-First is built on three beliefs:
- Evidence over narrative. A confident explanation is worthless without evidence. Every conclusion must be graded.
- Adversarial > affirmative. Finding reasons against your conclusion is more valuable than finding reasons for it.
- Fail-stop > fail-silent. When uncertain, stopping and asking is always better than guessing and continuing.
Use only Tier 1 + Tier 2 (4 files). This gives you 80% of the value.
Use all tiers. Add platform extensions as needed.
Fork this repo, add project-specific rules to platforms/your-platform/, and commit alongside your code.
See CONTRIBUTING.md for guidelines.
We especially welcome:
- Platform extensions — Android, React Native, Flutter, backend frameworks
- Adapter ports — new IDE/tool support
- Real-world examples — before/after cases showing verification in action
- Translations — help make this accessible globally
MIT — use it however you want.
- Andrej Karpathy — for articulating the core LLM coding pitfalls
- Addy Osmani — for pioneering the skill-based agent approach
- The Windsurf, Cursor, and Claude Code communities — for pushing AI-assisted development forward