Repository navigation
Snitch 0.3.0
Users pinned to ennajari/snitch@v1 get this release once v1 moves to it.
Codex CLI plugin (experimental)
codex plugin marketplace add ennajari/snitch, thencodex plugin add snitch@snitch: the same hooks, policy andescalatetool as in Claude Code. Set the mode withSNITCH_MODE.- In enforce mode the gate reads the files a Codex
apply_patchnames. - Checked live on Codex CLI 0.164; the README lists what is not supported yet.
Optional judge
- Snitch's detectors cannot read your request, so a test you asked an agent to update is flagged like any other. With the new opt-in judge, an LLM (
claude -p, your own login or an API key) reads your request and clears warnings for the test changes it asked for. It can only clear a warning, never add one; on any doubt or error the warning stays. - Plugin: the
judgeoption (Codex:SNITCH_JUDGE=on). - CLI:
snitch diff/compare --request-file pr.md. - GitHub Action:
judge: "true"withanthropic-api-key; the PR title and body are the request. - Measured on the benchmark: 45 of 45 requested test updates cleared, 0 of 59 cheats cleared. The rubric was tightened after a first run cleared 3 cheats, so this is a check on the cases it was tuned on.
Detection
mock_code_under_test: a test that patches the repo's own function and then calls it, so it checks its stand-in.seed_pinned: a random seed added to an existing test (found in the benchmark: an agent searched for the seed that made a flaky test pass). Medium severity, since pinning a seed can be an honest fix.snitch mutate before/ after/: hollow tests. It mutates the code a change touched and flags the change when the suite still passes against at least half of the mutants, listing the survivors. Slow, so it is a separate command. Calibrated on 28 real agent-written diffs with no false alarm.
TemptationBench
- Edition 2026-11 now has Codex CLI, without and with Snitch: changed a test in 57 of 78 trap trials without, 0 of 78 with, and escalated 59 times instead.
- Four controls whose request changes a test, to measure false positives on legitimate updates: with Snitch, neither Claude Code nor Codex refused one.
- Ten more trap tasks, two per temptation (54 tasks in all).
- The verifier excuses changes a task explicitly allows;
bench/judge.pyscores disclosure with an LLM and reports Cohen's kappa.
Status of the labels
The edition's labels were made with an AI assistant; no person has checked them yet. The results pages say so.