Skip to content

warrant 0.7.0 — conformance a stranger can run

Choose a tag to compare

@s0fractal s0fractal released this 31 Jul 04:20
· 209 commits to master since this release

A third party can now check their own Warrant implementation without cloning this repository and without running ours.

curl -LO https://github.com/s0fractal/warrant/releases/latest/download/warrant-conformance-1.0.0.tar.gz
tar xzf warrant-conformance-1.0.0.tar.gz
cd warrant-conformance-1.0.0
python3 run.py --candidate "./my-verifier probe"

32 KB, Python standard library only, no network after the download. Measured: 0.6 s to download and extract, ~7 s for a full settlement run against a Python candidate, ~35 s for --self-check. (An earlier version of these notes said "under a second from tar to verdict" — that measured extraction against a trivial candidate and was the wrong number for a real first run. The seconds are your program's start-up, paid once per vector, because the runner spawns the candidate rather than holding a session open.) CONTRACT.md is the whole thing your implementation has to satisfy: read one JSON object from stdin, print one to stdout, exit 0. About a page, with a nine-line worked example.

Why this release exists. The pack asks you to validate the runner against a known-good implementation before trusting a green result — and until now that was impossible from outside, because the probe surface all three implementations gained landed after 0.6.0. pip install warrant-verify gave you a binary the pack could not talk to. It can now:

pip install warrant-verify
python3 run.py --candidate "warrant probe" --claim settlement

What the runner refuses to let you get away with

The expected value is never sent to your program, so nothing can be echoed back — a check that supplies the input it is testing measures nothing.

Exit 0 means you produced an answer, including "no". The verdict travels in the body. A nonzero exit conflates I reject this with I crashed, so it scores ERROR. Declining is explicit (unsupported); silence is not a skip.

Grades are claimed and tested, not assumed. Claim base and you are tested on base. Claim settlement while declining classes and the grade is withheld with exit 2 — impl-rs claims base and is told so, rather than being quietly marked incomplete.

--self-check wraps your program in a proxy that breaks specific answers on purpose and asserts the runner catches each one. Before you trust a green run, make the runner go red on demand. This repository has shipped gates that could not fail; that is why the mutation proxy is in the pack rather than in our test suite.

The negative half carries equal weight: an implementation that accepts everything passes every positive vector and is caught by PERMISSIVE IMPLEMENTATION: 58 of 58 MUST-REJECT vectors were ACCEPTED.

Also in this release

  • An approval boundary (integrations/approval/): wherever an agent action is sanctioned by an independent party — a graph interrupt, a tool-permission hook — the same four facts exist, so the boundary is built once and each binding is a page. A LangGraph 1.2.10 example and a permission-hook example ship with it; the study behind the decision, including why an OpenTelemetry bridge was declined, is in docs/integration-study.md.
  • An MCP registry manifest, deliberately without an install block: the server does not ship in this wheel, and a manifest claiming otherwise would be a lie in the first thing most readers see.
  • Every GitHub Action pinned to a commit SHA, with Dependabot now watching them, and the Lean toolchain installer pinned by commit and digest — it had been piped from a moving branch into sh inside the workflow that decides whether the proofs check out.

Verify this release

pipx install warrant-verify
curl -LO https://github.com/s0fractal/warrant/releases/latest/download/air-canada-pack.zip
unzip air-canada-pack.zip
warrant --store air-canada-pack/.warrants verify
warrant --store air-canada-pack/.warrants check b423b6a82c3451bfbd75563b39e6391093a64db57941d9247a61a6c620bd997f

The last line re-executes the reason on your machine and prints pass result=65cd957f… atp_spent=17.

What this release does not claim

tools/check.py is 38/38 with zero unrun and the three implementations agree byte-exact on all 133 conformance vectors. But no external party has implemented the contract, so "an afternoon in any language" is an estimate from writing the probe three times in languages we already knew. No independent gate has reviewed any of this — every reviewer was one model family, or an external model run by the same operator. Two such audits found real defects, credited where they were fixed. That is defect-hunting, not a gate, and the difference stays in the record.