Skip to content

SignalBrain Agent Integrity

Choose a tag to compare

@whitestone1121-web whitestone1121-web released this 04 Jul 21:59

SignalBrain (pip install signalbrain) is an open-source CLI that scores AI coding agents on whether their claims survive re-execution. It centers on an “improvement receipt”: a small markdown file an agent writes beside each change, stating what changed, the shell command that proves it, and its confidence level. Any agent — Claude Code, Cursor, goose, Codex — can emit receipts using the open spec and paste-in prompt. After a human merge, sb score re-runs the receipt’s own proof command, records whether the claim held in an append-only JSONL ledger, refuses to score unmerged work, excludes same-PR test pins from trust, and preserves ledger order so history cannot be reshuffled. sb gate then computes trust by change class, granting auto-merge eligibility only when the last 10 high-confidence claims in that class hold at ≥95%, with CI-ready exit codes and a GitHub Action included. sb-mcp exposes receipt emission, validation, and gate status as MCP-native tools, and is listed on the official MCP Registry. Everything is local, inspectable, and telemetry-free: markdown receipts, JSONL ledger, no server, no account. Its anti-gaming rules were hardened through real attacks by the reference deployment’s own agents — padded streaks, self-scoring bypasses, and pin laundering — each resolved with published, git-reproducible rules. The result is rare calibration data: real agent confidence versus measured outcomes. In the first 58 measured claims, hold-rate actually fell as confidence rose — 86% at 0.85–0.90 confidence, but 33% above 0.95 — making the case that agent confidence should only be consumed through an earned track record. Apache-2.0: github.com/whitestone1121-web/signalbrain.