Repository navigation
v0.5.0
Minor Changes
-
The bench runner learns a repair loop. Pass
-loop Nto let one sample try up to N rounds: the harness runsrecord --dry-run --json, feeds the problem list back, and records only the last valid round. Round evidence lives underruns/<slug>/round-NN, the report gainsloops,fix_rates, andloop_cost, and-loop 1keeps the old single-shot files byte-identical. -
Findings can now carry a closed category (
logic,edge-case,security,
performance,accidental-commit,style,docs,other): an optional
per-comment label that never changes the finding's signature.report --categoryfilters to one class,report --check --categorygates only that
class, andreport,calibrate, andtrendall publish per-category counts
(untagged for unlabelled findings). Unknown or non-string values warn once
and store null; they never refuse a review. -
Findings can now carry a repair brief: the affected sites, a one-sentence
outcome, an optional approach and open questions.recordvalidates it against
the fetched change and keeps it across re-records,show --jsonandfix
print it, and it never changes a finding's signature. A malformed brief warns
and is dropped; it never refuses a review. -
verdict gateis one composed CI gate over the three dimensionsreport
already computes: open findings at-or-above a tier (--max-severity), a
completed post for the fetched HEAD (--require-head), and live unresolved
threads (--live auto|strict|skip). One exit code carries the answer — 0 pass,
1 fail-or-unknown, 2 usage, 3 tool error — and the human tail names each
dimension, the tally, and onetry:hint covering every failing dimension.
--jsonaddsdata.passedplusdata.dimensions[]infindings, head, live_threadsorder withcountnull where a dimension has no tally; a failed
gate keepserror: nullbecause a verdict is not a refusal.gateis
read-only (no store writes, no post) and absent from MCP: a local pass is not
authority to merge. -
Breaking (experimental MCP): a host on
verdict mcp --read-onlyno longer
sees athreadstool, andreportthere drops its--output/--sarif
arguments and reportsreadOnlyHint: falseinstead of claiming to write
nothing. Seven tools remain:report,next,show,trend,ready,
doctor,check-tree. A host that needs the thread feed should refresh with
verdict threadsoutside the server or run it without--read-only; a host
pinningreportas a read-only tool should stop trusting that hint. -
Patch identity dedupes re-reviews across rebases.
fetchandfetch-localwrite.verdict/state/<bundle>/identity.json(git patch-id --stable, labeleddiff-sha256fallback),postrefusesalready-reviewed-patchon a same-patch sibling unless--force(recorded inpost.jsonforced[]),adopt --from automatches exactly one local twin, andreport --completenesspasses when the posted and fetched heads differ only by a rebase with the same patch. -
verdict recordcan now answer "would this record?" before writing anything.
record --dry-runruns the same checks and writes nothing, returning a
summary of the comments, sections, and anchors that would move to the body.
Both modes share one checker instead of stopping at the first problem: the
refusal carrieserror.problems[]in document order with a JSONPath, a
publishedcode, the level, the message, the exactfix, and an optional
example, pluserror.problems_truncatedpast the 50-problem cap; human mode
lists the same problems before its try/re-enter lines. The checker owns the
brief, anchor/side, kb-id, chunk-attestation, confidence, security, claim,
triage, section, and verification-input checks, not just the top-level shape.
Warnings (moved anchors, brief parts dropped without refusing, malformed kb
ids, unknown fields, an unset rigor) let the record proceed unless--strict
or"strict_input": trueincontrols.jsonpromotes them.
pkg/verdict.CheckReviewexposes the pure checker, the MCPrecordtool gains
dry_run(served on a read-only server; a non-dry record is still refused),
and the review skill now loops write -> dry-run -> fix for at most three
rounds, then reports any remaining problems instead of recording a partial
review. -
verdict recordnow repairs the mechanical shapes weak models emit instead
of refusing them. Before checking,NormalizeReviewunwraps one
whole-document code fence, drops prose around exactly one balanced JSON
object, wraps a scalar where a string array belongs, renames the
inBounds/rung/filealiases, folds rigor synonyms and enum case, and
removes a BOM, an explicit null on an optional key, or an exact-duplicate
comment. Every rewrite is a warning with its own code, listed in
record.json'srepairs[];--strict,REVIEW_STRICT_INPUT=1, or
"strict_input": trueincontrols.jsonturns each into an error.
record --dry-runreturns the exact stored object asdata.normalized
plusdata.repairs, and never rewritesreview.jsonon disk. The CLI also
owns the body's> Basis:line, replacing a disagreeing line (or inserting
a missing one) with the computed counts and reportingbasis-replaced.
Unset rigor records assaidwithrigor_source: default, and length and
numeric bounds now refuse asout-of-rangeinstead ofbad-enum.
pkg/verdict.NormalizeReviewexposes the pure, idempotent primitive. -
Breaking: the published schemas moved with the skill, so every schema's
$idbase path is nowskills/verdict/schemas/rather thanschemas/—
https://github.com/stainless-code/verdict/blob/main/skills/verdict/schemas/envelope.schema.json,
for example. If you resolve a schema by URL, or pin$idin a validator, update
the base path; the schema contents are unchanged. -
The Go SDK and the TypeScript client can read review-thread state in-process,
so the one review read an embedder had to shell out for is now a method.
verdict.Threadswraps thethreadsverb:ThreadsRequestcarries the repo,
PR, and--provideroverride, andThreadsResponsereturns the envelope plus
the refreshed feed tally (Dir,File,Provider,HeadSHA,FetchedAt,
Truncated,Threads,Open,Outdated,Verdict), withData
byte-identical tothreads --json. The TS client gainsthreads(...)
resolvingEnvelope<DataFor["threads"]>, and every review verb now has a typed
client method instead of falling back torun(). No wire or behavior change:
the verb, its data profile, and its MCP tool already existed. -
verdict skill install --detectadds the review skill to every coding agent
already set up on the machine, instead of making you look up one path per agent.
It names each target and its result, and--jsonlists each target with its
status, path, and files written. With no agent set up it refuses and points at
--agent <name>, and--detectis mutually exclusive with--dirand
--agent.install.shandinstall.ps1now close a successful install with
that exact command. -
The TypeScript client's verb methods now return their own
dataprofile
instead ofEnvelope<unknown>:readyresolvesReadyData,fetchresolves
FetchData,trendresolvesTrendData, and so on for every verb method.
DataForexposes the same verb → profile map, so code that passesdata
around can name the profile without restating it.Every profile is generated as a
<verb>Data$defsentry in
envelope.schema.json(with the nested$defsit references), so non-TypeScript
clients get the same shapes from the published schema. Additions only:data
stays an open object that allows additional properties, no existing property,
required key, or type changed, andclient.run()still returns
Envelope<unknown>for verbs without a profile. -
Add typed configuration in
.verdict/config.json, with path scopes, captured explanations, review profiles, grouped spend and diff limits, and recorded deep validation. Local run artifacts now live under.verdict/state/; retired.review/knob files are no longer read.Cost records use schema version 2 with grouped lens and cost evidence. The Go Commander interface now takes a command pointer and preserves its working directory.
Patch Changes
-
Corpus cases accept an optional
also_lineslist of alternate anchors for
the same defect. Each entry is matched with the same line tolerance and
defect-naming rule as the primary line, matching any anchor counts the bug
once, and the reported line stays the primary one. The generated
corpus.schema.jsonnow requires positive line numbers inalso_lines
items, so schema validation rejects0and negative entries just as
calibratealready refused them ascorpus-invalid. -
calibrateandtrendgrade a corpus finding on the defect it names, not the
line it picked. A finding used to count as a true positive whenever its path
matched and its line sat withinline_toleranceof a known bug, so on a
four-to-seven-line hunk every anchor was correct and the reported precision
measured line-guessing rather than review quality; the same line could hold a
comment about variable naming and score as a catch. A true positive now also
has to name the bug: one of itskeywords(new, optional in
corpus.schema.json), itskind, or a phrase from the published category
vocabulary. The default window drops from three lines to one, and a findingrecordmoved out of its hunk is
never a true positive, so the anchor is measured and not assumed.The human false-positive line says which rule failed, so a run distinguishes a
finding that landed off the bug from one that landed on it and described
something else. Corpus bundles the benchmark runner writes now carry the
reviewedfull.diffand itshunks.tsv, sorecordscores against the hunks
the agent read instead of rebuilding an index from a diff that may have moved. -
The docs site publishes its documentation guide as the
verdict-docsskill
instead of a second skill namedverdict. Theverdictname belongs to the
review workflow —skills/verdict/SKILL.md, the payloadverdict skill install,
npx skills add stainless-code/verdict, and the embedded binary ship — and a
harness that indexed both well-known skills loaded whichever won, handing an
agent that asked how Verdict works the review workflow, or the reverse. The
site's/.well-known/agent-skills/index.jsonnow lists one skill,
verdict-docs, and the docs build fails if two published skills ever share a
name:. -
Global flags work anywhere on the command line — before the verb, between the
verb and the repo, or after the pr — soverdict --no-json report acme/app 9
andverdict report acme/app 9 --no-jsonprint the same output. A bare--now
ends positional parsing, so everything after it is a positional and
verdict fetch-local acme/app -- --jsonis a usage error instead of a stray flag. -
verdict mcp config --agent codex --writereads itsconfig.tomlthe way TOML
does, so averdicttable, key, or comment inside a string is text and no
longer a merge target: a file that ends inside an unterminated string, or whose
[mcp_servers.verdict]line opens one, refuses typed and keeps its bytes
instead of having keys appended into the literal. A table spelled inline or
dotted refuses rather than guess, the file's own CRLF or LF line endings are
kept, and an append no longer drops a trailing line of non-ASCII whitespace. -
verdict poston a PR that was never fetched now refuses with the typed
missing-bundleerror naming the fetch command, instead of failing internally
on its first write. -
Every release now publishes
commit.txt— the commit SHA the tag points at and
the certificate's workflow-SHA claim — andinstall.sh,install.ps1, and
verdict upgradepincosign verify-blobto it with
--certificate-github-workflow-sha; thesetup-verdictaction inherits the pin
throughinstall.sh. The identity and issuer alone accept any run of the
Release workflow, so a bundle from a different build of the same workflow used
to satisfy the check; the pin makes that a refusal. A release that publishes no
commit.txt, or acommit.txtthat is not a full 40-character SHA, is refused
withverify-failedinstead of being installed unpinned. -
A signed install no longer aborts when
cosignis on thePATH:install.sh,
install.ps1,verdict upgrade, andsetup-verdictnow verify the certificate
identity the Release workflow actually mints, which is a push tomainrather
than the tag the run publishes under. The old check asked for arefs/tags/…
identity that no release certificate ever carried. -
The shipped review prompts no longer hand an agent a corrupt skeleton to copy.
The "Technical details" block and the mermaid wrapper had a fence opened mid-line
with its body flattened onto it, plus orphan closers and an unclosed block, so an
agent following them emitted run-together headings and an unterminated fence.
Both are well-formed blocks now. -
npx skills add stainless-code/verdictnow installs just the skill instead of
copying the whole repository — over a thousand files — becauseSKILL.mdand its
payload moved intoskills/verdict/. That directory holds exactly the files
verdict skill installwrites. -
The skill now triggers on every review target it can handle — a pull request, a
branch, a commit range, or staged changes — so an agent picks it for local
reviews too.NOTICE.mdcarries the full MIT text of the upstream review
prompts Verdict adapts, pinned to the upstream commit, and the skill copy under
skills/verdict/ships that same license beside the prompts it covers. -
The TypeScript client's generated types render
anyOfas a union instead of
collapsing it tounknown, and a null variant folds into a trailing| null
the same way a"null"entry intypedoes.show'sdata.fix.verify_hint
now types asVerifyHint | nullrather thanunknown. -
verdict upgraderecognizes an install made withVERDICT_INSTALLeven after
the variable is gone, by reading the installer receipt beside the running
binary. A receipt is a hint, never proof of ownership: when npm, Homebrew, a
project'snode_modules, or a skill copy already owns the directory,upgrade
anduninstallkeep that channel and delegate or refuse with the owning command
instead of replacing or deleting a file the manager tracks. -
Shell completion now offers
threads, with its--providerand--json
flags. -
verifynow refuses a path glued to the option that takes it:tar -C/etc,
cp -t/etc/cron.d,sort -o/etc/x,grep -f/etc/passwd,gcc -I/usr/include,
git -C/etc, andgo build -o/etc/xexit before the command runs instead of
executing it. The pre-exec gate only recognized a path after a space or a
separator, so every spelling getopt accepts — a clustered-czf/etc/x, a
positional-Cafter--, a quoted or backslashed-C/etc, an uppercase
-C/ETC— reached the shell with the escape unreported. An attached value is
now judged as the path it is, and the refusal names the fragment it read.One spelling to know: a relative value that starts with a letter and contains a
slash (git -Ctools/cli) has the same shape as-C/etc, so the gate refuses
it rather than guessing. Writegit -C tools/cli— both spellings of a relative
value stay allowed once a space separates the option from its value. -
A canceled re-verify keeps the last completed proof, and a failed Windows handoff leaves no staged helper behind.