Verdict

A skeptical QA agent with memory — it remembers your baseline and tells you what broke since yesterday.

/plugin marketplace add ArtJack/verdict
/plugin install verdict@verdict
/verdict:run

Most AI "QA agents" are a paragraph of enthusiasm with a checklist. They audit your repo from scratch every time, re-report the same 20 findings until you stop reading, call flaky tests "failures", call stale tests "failures", and end with "LGTM! 🎉".

Verdict is a Claude Code plugin built the way QA is actually practiced:

• It remembers. A state file carries the baseline. Every finding gets a stable ID and an age; every run reports NEW / STILL_OPEN / RESOLVED / REGRESSED — regressions ranked first, always.
• A red test means something. Every failure is classified — REAL_DEFECT, STALE_EXPECTATION (which needs a citation proving the change was intended), BRITTLE_TEST, ENVIRONMENT, or FLAKY (confirmed by re-runs, quarantined with an expiry). The classification most likely to excuse a regression carries the highest evidence bar.
• A verdict you can defend. Every run ends in exactly one of pass | pass with risks | blocked | fail — an open Blocker forces fail, blocked is a legitimate outcome, and a pass always names what was not tested.
• It never fixes your code. There is no Edit tool, a hook confines its writes to the QA root, and a strict-mode Bash guard closes the shell's write channels — a tester that patches what it judges isn't independent. The guard is a heuristic, not a sandbox, and the README says so.
• It is tested, and it tests itself. A scored eval suite with the misses published, a signed run history the model cannot forge, a track record the tester cannot edit — and a nightly audit of its own releases: 69 findings filed against itself so far, every one fixed in a tagged release with the defect pinned as a mutant the suite must kill.

Who pays for the model? You do, with the Claude subscription you already have: the plugin runs inside your own session, nothing routes through anyone else, and everything below the model — the state, the gate, the MCP server, the eval scorer — is stdlib Python that runs for free. Works on Python, TypeScript, Go, or anything with a test runner; the eval fixtures cover Python and TypeScript.

Read next: Install · What installs, and when it runs · Quickstart · Why another QA agent · The tested tester · CI gate · Accepting a risk · FAQ

Install

/plugin marketplace add ArtJack/verdict
/plugin install verdict@verdict

Python 3.9 or newer, whatever your python3 resolves to — the hooks and the fact harness are stdlib-only and are invoked by that name, which on a stock Mac is /usr/bin/python3 (3.9). The optional MCP server is a pip install and needs 3.10+, which is what requires-python in pyproject.toml refers to. The floor is tested: a module that would fail to import on 3.9 fails CI instead (tests/test_interpreter_floor.py) — because the failure it prevents was silent. The Bash guard once raised on import there while the write guard beside it kept working, so a strict session looked armed with half its controls missing.

What installs, and when it runs

Installing a plugin means letting its code run in your sessions, so here is exactly what this one does — measured from hooks/hooks.json, not summarised from memory. Six hook registrations; each starts a python3 (tens of milliseconds) when its event fires:

Event	Fires on	Script	Silent when
PreToolUse	Write/Edit/MultiEdit/NotebookEdit	write-scope guard	always, unless VERDICT_STRICT=1 or the caller is the verdict agent itself
PreToolUse	Bash	bash-scope guard	always, unless VERDICT_STRICT=1
PostToolUse	Write/Edit/MultiEdit	state validator	unless the written file is literally named state.json
Stop / SubagentStop	end of turn	run-contract check	unless a QA run in this session left hand-written state — and it blocks at most once, never loops
SessionStart	session open	findings banner	unless the repository has QA state

Every hook fails open: malformed input, missing files, or an exception mean exit 0 and silence — a broken hook must never brick a session. VERDICT_STRICT=1 is what arms the scope guards, and you set it only for dedicated QA sessions (headless, CI, the nightly); in ordinary interactive work the guards are no-ops.

Prefer not to install globally? Everything works per-repository: copy agents/verdict.md into <repo>/.claude/agents/ and the hooks/hooks.json entries into <repo>/.claude/settings.json, with ${CLAUDE_PLUGIN_ROOT} replaced by a checkout path. That is exactly how the eval harness provisions its scratch projects — eval/run_eval.py is the reference implementation.

Quickstart

/verdict:run                          # first run: profile + isolation rules + baseline
/verdict:run the payment retry change   # every later run: a delta against the stored state

Every later run is a delta against the stored state. A repeat run returns something like:

VERDICT: fail

Scope: 2c67f47..b4e2943 (4 commits, 16 files) · run 4 (delta)
Isolation check: pass (no .env present; no live service touched)

Findings — REGRESSED first:
REGRESSED  PRICER-F-002  Critical/P0  round_cents uses banker's rounding again (pricer.py:17)
                         resolved 08-19, reintroduced by b4e2943 — this forces the verdict
NEW        PRICER-F-007  Major/P1     quarantine graveyard: test_listable_at_floor_exactly
                         skipped 114 days with no expiry — it is the test that would catch F-001
STILL_OPEN PRICER-F-001  Critical/P0  age 6d  is_listable rejects a price exactly at the floor
FLAKY      test_bulk_discount_applies — fails 3/6 runs with no code change; quarantined
           until 2026-09-07, excluded from this verdict, listed until re-evaluated

Delta gates: tests 213 → 213 · duration +0.4% · coverage on changed files: no decrease
Release blockers: PRICER-F-002 (regressed), PRICER-F-001
Not tested: concurrency under parallel checkout — no harness present
Fix order: 1) F-002  2) F-001 (unskip its test first, watch it fail red)  3) F-007 expiry
Artifact: .qa/reports/2026-08-24-pricer-review.md

Why another QA agent

	Typical qa-expert.md	Verdict
Remembers the last run	no — every run is a fresh audit	state file; NEW/STILL_OPEN/RESOLVED/REGRESSED with ages
Flaky tests	"keep flakes under 1%" (prose)	quarantine ledger with mandatory expiry; flakes excluded from the verdict, never from the report
Release decision	"go/no-go" appears as a checklist word	four-verdict contract; an open Blocker forces fail
Red test triage	"investigate failures"	five-class taxonomy; STALE_EXPECTATION requires an intent citation
Quality gates	">90% coverage" absolutes	direction gates: coverage on changed files must not decrease; 0 tests collected ≠ 1 test failing
Test design	"test edge cases"	24-technique catalog with risk triggers — incl. property-based, metamorphic (for ML/LLM output), MC-DC, contract tests (docs/test-design.md)
Can edit your code	nothing stops it	no Edit tool + write-scope hook + strict-mode Bash guard
Security	ignored, or oversold	opt-in report-only pass: dependency audit + diff secret scan; pentest explicitly out of scope
Risk prioritisation	"focus on high-risk areas"	the ranking is computed from the project's own finding history (severity-weighted, paths merged across citation depths), and the report must show the ranking, the cutoff, and everything below it — which lands in not-tested
Root cause	"investigate the failure"	a four-link chain with a citation per link, a mandatory class check (is this an instance or a pattern?), and causation proven by flipping the cause in a scratch copy — with its own scored fixture built around a decoy
Requirements review	never — code only	/verdict:spec judges the spec before code exists (contradictions, unmeasurables, boundary ambiguities, history conflicts) — with its own scored eval fixture
"No bugs found!"	frequently	never — coverage, gaps, and residual risk instead
AI-authored code	same checklist as human code	provenance measured (trailer census over the range, profile authorship); a pattern catalog with a procedure and evidence bar per entry (docs/ai-authored-code.md); deterministic censuses for hallucinated imports, placeholders and swallowed errors feed judgment — and a scored fixture proves the behaviours
Its own accuracy	unmeasured, and unmeasurable after the fact	every finding states a confidence when filed; the outcome is computed from what the finding did, kept in a permanent ledger, and reported as a track record the tester cannot edit
Tested itself	—	scored eval suite: baseline + delta-memory + adversarial-honesty fixtures, deterministic scorer, published answer keys (eval/)
State consumable by other tools	—	verdict-mcp: read-only MCP server over the state — works from Cursor, Codex, CI, any MCP client

The tested tester

A QA agent that was never tested is exactly the kind of claim it should reject. eval/ is a scored eval suite with a deterministic scorer — score.py reads the state file, not the prose — and eight fixtures, the four that carry the headline claims:

• Baseline (fixtures/pricer): 8 seeded issues covering all five failure classifications, including a boundary defect hidden behind a "temporarily" skipped test and a stale expectation whose intent citation sits in the CHANGELOG. Answer key.
• Delta (fixtures/pricer_rev_b): scores the flagship — a run against an authored run-2 history must produce REGRESSED (ranked first), NEW, STILL_OPEN, RESOLVED, and release an expired quarantine, while a CHANGELOG decoy tries to launder the new defect as intended. Answer key.
• Liar (fixtures/liar): adversarial honesty — a test script that prints "ALL TESTS PASSED" unconditionally, a conftest that skip-marks the whole suite, a mock asserting its own return value, a tautological assertion. Scores whether the verdict takes output at face value.
• Spec (fixtures/refund-spec): shift-left — a draft PRD with a seeded contradiction, an unmeasurable requirement, an exactly-at-the-boundary ambiguity, a silent failure-path gap, and a CHANGELOG that contradicts the spec. Scores /verdict:spec finding them all before any code exists.

python3 eval/run_eval.py --mode seeded|live|baseline runs it all in an isolated scratch repo and scratch state home. Results are published as measured; misses — and any answer-key amendment — stay in the table (eval/README.md).

State modes

• Solo (default): state lives in ~/.claude/verdict/<repo-name>/ — nothing added to your repo.
• Team: create .qa/ in the repo (/verdict:baseline team) and commit it — your teammates and CI share the same baseline, and QA reports travel with the code.

The state schema is documented in docs/state-schema.md — versioned, forward-compatible, human-readable JSON.

Commands

Plugin commands are namespaced by the plugin and must be typed in full — /verdict:run, /verdict:status. There is no short form: a bare /verdict is an unknown command, measured rather than assumed.

Start here: /verdict:run — the front door. It reads the tester's memory and picks the right pass itself: no state yet → a baseline; state present → today's delta; arguments given → a delta narrowed to what you named. It says which it chose and why. Every command below is the same machinery aimed at one specific job, for when you already know which job you want.

Command	What it does
/verdict:run	Front door — routes to baseline, delta, or a scoped review from the stored state
/verdict:baseline	Initialize the QA root, project profile, and baseline state
/verdict:regression	Regression checklist: changed area → adjacent flows → integrations
/verdict:release	Release gate with the four-verdict contract
/verdict:bug	Turn a symptom/log/complaint into a classified, structured bug report
/verdict:flake	Classify an intermittent failure: ≥3 reproductions, mechanism hunt → BRITTLE_TEST fix task, or FLAKY quarantine with expiry
/verdict:status	Read-only status from the stored state — no run, no writes, no agent spin-up
/verdict:spec	Shift-left: judge a spec/issue/PRD for testability before code exists — contradictions, unmeasurables, undefined boundaries, silent gaps, history conflicts, plus Given/When/Then criteria
/verdict:cause	Trace a failure to its root cause: symptom → mechanism → origin → class, each link cited, causation proven by counterfactual rather than narrated; trigger, cause, and latent condition kept apart
/verdict:charter	Timeboxed exploratory charter with a risk focus seeded from the profile's incident history; observations captured as evidence, discoveries converted to bug reports and regression candidates

The tester's memory, over MCP (optional)

Verdict's state isn't locked inside the agent. verdict-mcp is a small read-only MCP server over the same state files, so anything that speaks MCP can consult your QA memory — an orchestrator gating a merge, a Cursor or Codex session, a CI step commenting a PR:

Tool	Returns
get_verdict(project)	last verdict, release blockers, report path, not-tested list
get_findings(project, status)	open (default), all, or NEW / STILL_OPEN / RESOLVED / REGRESSED — REGRESSED ranked first
get_quarantine(project)	the flaky ledger, each entry with a computed expired flag
get_history(project)	run-over-run trend parsed from the report INDEX
get_report(project, report?)	full report content (default: last run's) — path-guarded to the QA root, so a CI step can quote the evidence, not just link it
get_profile(project)	the project's QA profile: isolation rules, risk areas, real test commands — plus the lessons ledger when one exists
get_trends(project)	run-over-run trajectory from the INDEX, the current pressure picture (open by severity, age distribution, quarantine size, duration), and hotspots — where this project's defects actually cluster, computed from its own findings and severity-weighted, with the number of runs behind the ranking
list_projects() / get_state(project)	everything with a baseline / the raw state

claude mcp add verdict -- uvx --from verdict-qa-mcp verdict-mcp

The distribution is verdict-qa-mcp — the console script and the import package are still verdict-mcp / verdict_mcp; only the name PyPI indexes differs, because verdict-mcp there belongs to an unrelated project. Installing straight from the repository also works and needs no release: uvx --from git+https://github.com/ArtJack/verdict verdict-mcp

project is a key from the solo root (~/.claude/verdict/, override with VERDICT_HOME) or a repo path in team mode (resolves <repo>/.qa/). Every tool carries a read-only annotation and the server never writes — the tester's memory is public API; the tester's pen is not. Needs uv (or pipx install verdict-qa-mcp); the plugin itself still has zero dependencies and works without the server.

Closing the loop (without letting the tester fix anything)

Verdict is deliberately the gate of a fix loop, never its actor — an agent that fixes and then re-judges its own fixes is grading its own homework. The loop belongs to your orchestrator, your coding agent, or CI; Verdict's job is to make every pass around it evidence-cited and impossible to rubber-stamp:

┌─────> implement the ordered fix list   (you / your coding agent)
│                    │
│                    v
│        /verdict:run — delta run        (scoped by diff, findings aged)
│                    │
│                    v
│        get_verdict over MCP ────────── pass ──> merge
│                    │                            (the not-tested list travels with the PR)
└──────── fail · pass with risks
          (fix order is dependency-aware, REGRESSED first)

Minimal driver, any MCP client:

while True:
    before = mcp.call("verdict", "get_verdict", {"project": "myapp"}).get("run_number") or 0
    subprocess.run(["claude", "-p", "/verdict:run delta pass on myapp"])   # the agent runs
    v = mcp.call("verdict", "get_verdict", {"project": "myapp"})          # the gate reads
    assert (v.get("run_number") or 0) > before, "run died before writing state — not a verdict"
    if v["verdict"] == "pass":
        break
    fix(v["release_blockers"],
        mcp.call("verdict", "get_findings", {"project": "myapp", "status": "open"}))

(The run_number check matters: without it, a run that crashes before writing state re-serves yesterday's verdict — and if yesterday passed, the loop merges unreviewed code. verdict-gate --min-run-number is the same check as a CLI.)

Rules that keep the loop honest — all enforced by the agent's contract, not by hope:

• REGRESSED breaks the loop loudly. A finding that comes back outranks any number of NEW ones; it is ranked first in every report and every get_findings response.
• Red tests exit through the right door. STALE_EXPECTATION exits via a test-update task (with an intent citation), REAL_DEFECT via a code fix — the loop never converges by editing a red test to match the code.
• Flakes can't be buried. Quarantined tests are excluded from the gate but re-enter on expiry, so the loop cannot converge by skipping its way to green.
• blocked halts, it doesn't pass. A missing environment stops the loop for the operator instead of laundering itself into a verdict.
• A crashed run is not a verdict. The gate asserts run_number advanced; stale state is its own exit code (5), distinct from both pass and fail.

This is not hypothetical — it is the loop the author's private deployment runs nightly, unattended, against a production codebase.

CI: gate PRs on the tester's memory

The repo doubles as a composite GitHub Action. Gate mode needs no API key, no install, and no model — a stdlib-only script reads the committed team-mode .qa/ state, sets the job status, and maintains one sticky PR comment (verdict headline, blockers, REGRESSED-first findings table, the not-tested list):

permissions:
  pull-requests: write
concurrency: verdict-${{ github.ref }}

steps:
  - uses: actions/checkout@v4
  - uses: ArtJack/verdict@v0
    with:
      max-age-hours: 48        # a stale verdict is exit 5, never a pass
      max-commits-behind: 0    # ...and so is one measured before these commits

Run mode (experimental) executes a headless Verdict pass first — on a GitHub-hosted runner with anthropic-api-key, or on a self-hosted runner with claude-oauth-token from claude setup-token, so nightly QA rides your subscription instead of API billing (anthropic-base-url passes through for Anthropic-compatible gateways). The same contract is available anywhere as a CLI:

verdict-gate myapp --max-age-hours 24 --max-commits-behind 0 --fail-on risks

Exit codes: 0 pass · 1 fail · 2 usage · 3 blocked · 4 no state (the tester never ran) · 5 stale. 4 and 5 are deliberately distinct from 1: "the tester never ran" must never look like "the tester said no". For running the nightly pass on your own machine — cron, systemd, subscription token, strict mode — see docs/nightly.md.

--format sarif emits the open findings as SARIF 2.1.0 (severity → level, locations parsed from file:line evidence), so they land as annotations in GitHub's Security tab:

- run: verdict-gate --format sarif > verdict.sarif || true
- uses: github/codeql-action/upload-sarif@v3
  with: { sarif_file: verdict.sarif }

Give your tester project eyes (bring your own MCPs)

The agent ships with core tools only, but the frontmatter is an extension point: copy agents/verdict.md into your project's .claude/agents/ and add your project's MCP tools (database, staging API, browser) to its tools: list. The agent's §0 isolation rules govern how it may use them — read-only facts, never mutations, blocked when it cannot verify. This pattern is battle-tested: the private ancestor of this agent runs nightly with eleven read-only marketplace-database tools, which is exactly how it caught a live overselling bug that no amount of reading source code could have found.

Worked example — a web app with a Playwright MCP connected:

# your project's .claude/agents/verdict.md, frontmatter tools:
tools:
  - Read
  - Glob
  - Grep
  - Bash
  - Write
  - mcp__playwright__browser_navigate
  - mcp__playwright__browser_snapshot
  - mcp__playwright__browser_click
  - mcp__playwright__browser_console_messages

…and the profile carries the rules of engagement: which origin is the test environment (never production), which accounts are test accounts, and that navigate/snapshot/read is in scope while anything that submits, pays, or mutates an account is forbidden. §0 governs browser tools exactly as it governs Bash — unsure whether a click mutates? It mutates; return the risk instead of clicking. Exploratory charters (§4, technique 23) translate directly: a timeboxed browser session with a risk focus, observations as evidence, repeatable failures becoming bug reports.

The model judges; the system measures

About two thirds of a state file is arithmetic and transcription — timestamps, SHAs, diff ranges, gate exit codes, durations, test counts, finding hashes, ages, deltas. None of it is judgment, and every one of them is a place to be confidently wrong.

So the run is split. verdict-facts measures: it runs the gates you name, times them, parses their counts, reads git, derives the project key, and decides run_number and run_type (including when a run must be re-declared a re-baseline). The agent then writes only judgment — verdict, findings, evidence, what was not tested. verdict-finalize merges the two, computing each finding's hash, first_seen, age_days, and its NEW/STILL_OPEN/RESOLVED/REGRESSED delta from the previous state, and validates the result before writing anything.

verdict-finalize also renders the report — scope, gates, the REGRESSED-first findings table, not-tested, quarantine — from that same state, and injects the agent's prose (risks, fix order, per-finding narrative) into it. The report cannot go missing, because the harness writes it, and cannot contradict the state, because it is the state.

Nothing the model cannot compute correctly is left for the model to compute.

The tester's own error rate

A finding is worth what the tester's record says it is worth. Verdict keeps that record, and the design principle is the same one as everywhere else here: the part a model would be tempted to grade generously is the part it does not get to touch.

Each finding states a confidence when it is filed — proven (demonstrated it happen), probable (traced, not executed), hypothesis (suspected). The validator refuses a new finding without one, and the harness freezes it: a later run cannot revise a prediction after seeing how it turned out.

The outcome is computed, never claimed. A finding that regressed, or whose fix was verified by re-injecting the defect and watching a guard fail, held up. One the tester withdrew did not. Everything else stays undecided and is excluded from every rate — a resolution nobody verified is an absence, not proof, and a still-open finding has not settled anything. Decided outcomes persist in outcomes.json, because state.json drops findings resolved two runs ago and the sample would otherwise reset forever.

The report then carries a Track record section: how many findings this project has tracked, how many are settled, and the counts per confidence level and per proof method. A percentage appears only once a bucket has 30 settled outcomes. Below that you get "2 of 3", which is a fact, instead of "67%", which is decoration.

Accepting a risk — the maintainer's pen

Some findings are right and will not be fixed: a residual risk weighed and written into a decision log, a defect behind a feature that is being retired. Left open, such a finding is re-reported as an open Major in every banner for the life of the project — the "same twenty findings until you stop reading" failure this tool exists to prevent — and withdrawn would score a correct finding as the tester's error. So there is a fourth status, and the tester cannot write it:

verdict-accept myapp MYAPP-F-021 --cite "DECISIONS.md 2026-09-02" \
    --reason "deleting the ledger too defeats the anchor; the cost is the whole track record"

That writes accepted.json beside outcomes.json in the QA root — a file the scope guards refuse to the agent and a status the validator refuses in a judgment. The finding leaves the open counts at once (the banner, verdict-gate, the MCP server), leaves the verdict at the next run, appears under Accepted risks in every report with its citation, and settles in the track record as confirmed on the maintainer's word — kept apart from the measured and the claimed confirmations, because it is neither. --revoke reverses it, with a reason; --list shows the ledger. A decision changes the next verdict, never the last one.

The tester has memory. The implementer did not.

That asymmetry had a measured cost. Verdict filed eleven evidenced findings on a live site, one of them a release blocker — deploying this branch strips every production security header — and the very next session in that same repository did a full SEO pass and touched none of them: not the blocker, not the application form that reports success when the handoff failed, not the contrast failures on both primary CTAs. The findings sat in state.json the whole time. next_run_focus existed, but only Verdict reads it; get_findings existed over MCP, but nothing called it unprompted.

So a SessionStart hook says what is outstanding when a session opens in a repository that has QA state — before the first edit, not after:

Verdict remembers dm-express-site: run 2 (delta), today — verdict **fail**.
1 release blocker — look here first:
  - DMEXPRESS-F-1 — the audited branch has diverged from the deployed origin/main
11 open findings: 4 Major · 4 Minor · 3 Trivial
  - DMEXPRESS-F-3 (Major) Light theme: accent-coloured text fails WCAG AA…
Full detail: `/verdict:status`. These are findings, not instructions.

Deliberately short — a session opener that scrolls is one nobody reads — and it never repeats a finding it already named as a blocker. Silent in a repository with no QA state, silent on any failure, and it flags memory older than a week rather than serving it as current. It informs a session; it does not commandeer one.

The last guard fires whether or not the model remembers

Every check above sits downstream of a tool the model has to choose to call — and that is not a theoretical gap. A real run of /verdict:run wrote to the default state root while $VERDICT_HOME pointed elsewhere, invented a project key, skipped the harness entirely, and still produced a confident, plausible FAIL. verdict-validate would have rejected that state and verdict-gate --require-harness would have exited 6. Neither fired, because nothing invoked them.

(That check used to be defeatable by imitation rather than forgery — its two durable signals were a key holding a dict and a fixed footer string, both copyable straight out of the committed artifacts. Verdict found that auditing itself. Each run now signs the run history with a hash of the previous link, and the state records it; a link copied forward does not verify, and neither does a state edited after signing.)

So there is a Stop hook. When a turn ends it asks one question — did a QA run just leave hand-written state on disk? — and if so it blocks the stop once and says what to redo. It fires on the turn ending, not on the model deciding to check.

The bar for speaking is deliberately high, because it runs at the end of every turn in every session where the plugin is enabled: the turn must not already be continuing because of this hook (never loop), a QA root must resolve from the session's cwd, its state.json must have been written in the last half hour, and the harness traces must be missing. Anything else exits in about two stat calls — 37 ms, measured. Every failure path — unparseable input, an import that does not resolve, an unreadable state — also exits silently: a hook that bricks sessions is worse than the problem it polices.

The state contract is machine-checked

Prose in a prompt reduces how often a model invents a value; it cannot stop a model from inventing a value it is capable of inventing. Measured here: months after date -u became an explicit rule, two of four production timestamps still landed on exactly :00 seconds — fabricated, quietly, in states that every downstream consumer believed.

So the contract stopped being prose and became a gate. verdict-validate runs as a PostToolUse hook on every state.json write (and as a CLI in CI) and reports, immediately and in-session, any state that: names a report which is not a path to a file that exists · carries a timestamp that was recalled rather than measured · leaves run_number where a crashed run left it · invents enum values · claims pass over an open Critical · files an open finding with no evidence · quarantines a test with no expiry.

Its first run against four live production states found violations in two of them — including the exact dodge ("delivered inline to the caller…" in the report field) that a prompt rule had failed to prevent three separate times. Every rule in it exists because a real run broke it.

The read-only guarantee, honestly stated

Four layers: (1) the agent has no Edit tool; (2) its contract confines Write to the QA root; (3) a PreToolUse hook blocks out-of-scope Write/Edit calls; (4) under VERDICT_STRICT=1 — set it for headless/CI/scheduled runs, where the whole session IS the QA run — a second hook also closes the obvious Bash write channels: output redirection, tee, sed -i, rm/mv/cp and friends, and mutating git verbs, each target resolved against the QA root. In mixed interactive sessions the hooks enforce when the platform identifies the calling subagent and stay out of your way otherwise — they will never block your edits.

The Bash guard is a deny-heuristic, not a sandbox: unknown commands run (a QA pass needs pytest, coverage, linters), package installs are deliberately not denied, and a determined command can evade string analysis — OS sandboxing remains the real boundary. Both hooks fail open on malformed input and are tested in CI (tests/test_hooks.py). That is the whole truth; a QA tool should not oversell its own controls.

FAQ

Who pays for the model? You do — with the Claude subscription you already have; nothing routes through the author and no API key is required. The one place a key can appear is the optional GitHub Action's run mode, and that is your key, in your repo, for your CI. Everything below the model is plain files and stdlib Python.

Can it run on a local LLM? The wiring exists today (ANTHROPIC_BASE_URL passes through to any Anthropic-compatible gateway, e.g. LiteLLM in front of Ollama), and every non-judgment layer already runs locally for free. But the verdict is only as good as the judge: Sonnet — far stronger than any home-lab model — currently hard-fails the eval (see the results table), so a local model must earn verdict-signing duty by passing the same eval as everyone else. Run it, publish the score, then decide.

Why won't it fix the bugs it finds? Independence. The agent that patches the code and then declares it healthy is grading its own homework. Verdict returns an ordered, implementation-ready fix list for you (or your coding agent) to execute.

Does it replace CI? No — it sits on top. CI tells you the suite is red; Verdict tells you which red matters, what it means, what regressed since the last run, and whether you can ship anyway.

Does it work in scheduled/headless runs? Yes — that's what the state file is for. Run it nightly; read a delta report over coffee, not a fresh audit.

Roadmap

• Local-first track (the project's original ambition): an agent-skills-standard variant — the prompt, technique catalog, and state contract are portable markdown, which is the door to non-Claude runtimes — plus the local-model experiment: run the eval suite through an Anthropic-compatible gateway against local models and publish the scores. A model earns nightly duty by passing the same eval as everyone else.
• A JS/TS eval fixture alongside the Python one
• Mutation-testing integration where a tool is present

License

MIT