Repository navigation
v26.10.2
Highlights
- Eval critics in one round now share mechanical-check evidence instead of re-running identical commands: the first critic to run the unit suite, lint, build, typecheck, or a scripted e2e suite logs it to a per-round ledger, and sibling critics cite that entry rather than paying for the same run again. Rounds that previously spent N × suite-time on the same fixed code now spend roughly 1×, in both wall clock and tokens.
Changes
tenet_start_evalinserts a Shared Evidence Ledger block into every critic's prompt — built-ins and customs, grounded or ungrounded — when the evaluated job belongs to a run. The ledger is one append-only JSONL file per eval dispatch:.tenet/runs/<run-slug>/critics/evidence/eval-<source-job-id>-<eval_round>.jsonl. Every new dispatch (source retry, re-fire, contention re-run) starts a fresh file, so evidence never leaks across rounds.- Citation is skeptical by design: a critic cites an entry only if its
output_excerptplainly shows the outcome itsexit_codeclaims, and only for what the excerpt supports (a report needing per-test detail re-runs). Conflicting, unparsable, or doubtful entries are re-run; if a critic observes a mid-round code change (HEAD moved past the evaluated commit, which the block embeds as an anchor), every entry is treated as stale and re-run. - The ledger is best-effort and never a gate: no locks or coordination (concurrent duplicate runs are accepted), ledger IO problems never fail or stall an eval, entries record mechanical facts only and never a pass/fail verdict, and a critic that finds any rule unclear simply runs every check itself. Jobs that don't belong to a run (ad-hoc jobs) are unchanged.
- Docs:
skills/tenet/critics.md(what custom-critic authors should assume, plus the retry caveat),skills/tenet/phases/06-evaluation.md(ledger semantics and the Stage 1 dedup note), and CLAUDE.md.tenet/conventions.
Breaking changes
none