Grade your agent fleet against rules it must never break.
New: modules/eval-harness/ (v0.1.0)
Turns a multi-agent event log into reproducible traces and scores them against explicit safety invariants. Zero LLM tokens, pure stdlib, and the evaluator's exit code is the number of failed checks.
Four deterministic behavioural invariants:
- a human gate before a risky commit,
- an independent verify before commit,
- no duplicate-event storms,
- escalations get resolved.
Two benchmarks built from six weeks of our own production fleet: a readable showcase (public-live-v0) and the full 317-event corpus (consensus-safety-v0).
We publish our own failure alongside our own pass: human gate before a risky commit 100%, independent verify before commit 7.7%. The second number is bad, it is ours, and a harness that only ever reports the good invariant is a marketing asset rather than an eval.
Sanitisation ships as two deterministic paths — a structure-only whitelist, and allowlist curation that keeps real text — and the host role map is local config, never source.
Also in this release
AGENTS.md— determinism andsanitize.pyare the two load-bearing rules for anyone, human or agent, changing this repo.FOR-ROBOTS.md, changelog categories for these auto-generated notes, and the contributor deal inherited from one org-wideCONTRIBUTING.mdinstead of a local copy that silently shadowed it (we found six repos carrying the stale copy and deleted them).
About the timing
The module landed on 29 July and sat in Unreleased for six days while the README already described it as shipped. Under the release discipline this repo now follows, a shipped module is a release — cut when the work lands, not when someone remembers.
What's next
A Second Brain reference implementation (vault + RAG + reranker), the graph/associative recall layer with entity-vs-theme gating, the C(H+A)RM relationship layer, and the import pipelines. The release feed is how you tell what is real here from what is still a manifesto.
Full Changelog: v0.3.0...v0.4.0