Skip to content

v26.10.2

Choose a tag to compare

@github-actions github-actions released this 19 Oct 03:27
· 3 commits to main since this release

Highlights

  • Eval critics in one round now share mechanical-check evidence instead of re-running identical commands: the first critic to run the unit suite, lint, build, typecheck, or a scripted e2e suite logs it to a per-round ledger, and sibling critics cite that entry rather than paying for the same run again. Rounds that previously spent N × suite-time on the same fixed code now spend roughly 1×, in both wall clock and tokens.

Changes

  • tenet_start_eval inserts a Shared Evidence Ledger block into every critic's prompt — built-ins and customs, grounded or ungrounded — when the evaluated job belongs to a run. The ledger is one append-only JSONL file per eval dispatch: .tenet/runs/<run-slug>/critics/evidence/eval-<source-job-id>-<eval_round>.jsonl. Every new dispatch (source retry, re-fire, contention re-run) starts a fresh file, so evidence never leaks across rounds.
  • Citation is skeptical by design: a critic cites an entry only if its output_excerpt plainly shows the outcome its exit_code claims, and only for what the excerpt supports (a report needing per-test detail re-runs). Conflicting, unparsable, or doubtful entries are re-run; if a critic observes a mid-round code change (HEAD moved past the evaluated commit, which the block embeds as an anchor), every entry is treated as stale and re-run.
  • The ledger is best-effort and never a gate: no locks or coordination (concurrent duplicate runs are accepted), ledger IO problems never fail or stall an eval, entries record mechanical facts only and never a pass/fail verdict, and a critic that finds any rule unclear simply runs every check itself. Jobs that don't belong to a run (ad-hoc jobs) are unchanged.
  • Docs: skills/tenet/critics.md (what custom-critic authors should assume, plus the retry caveat), skills/tenet/phases/06-evaluation.md (ledger semantics and the Stage 1 dedup note), and CLAUDE.md .tenet/ conventions.

Breaking changes

none

Full changelog

v26.10.1...v26.10.2