Skip to content

cedr v0.8.0

Choose a tag to compare

@github-actions github-actions released this 02 Nov 05:12
· 9 commits to main since this release

cedr v0.8.0: bench

v0.8.0 · 27 changes from 1 contributor

Highlights

Bench is here. It tells you whether a change to your rules, a skill or a model helps before you merge it: real tasks against a baseline and a candidate, your tests deciding each trial, and a budget on every run that defaults to $2.00. Start with cedr bench init.

Route puts you in charge. Set its plan limits per plan and per route, send different work sizes to different models, and change all of it from the command line. Route and Council can also read Grok Build's plan usage and balance work onto it.

Guard watches remote MCP servers. A remote server has no version, so a pin could never notice its tools changing; cedr guard remote compares what a server lists now with what you locked.

Upgrade step: none; cedr updates itself. Upgrade every machine that shares a routes.toml before using Route's new keys in it, because older versions refuse them. A lock made with --remote needs this version or later to diff its surfaces.

Features

  • Bench adds cedr bench init, plan, run, history, show and budget. It runs the unmodified claude and codex CLIs, and any other agent through a command harness, each trial in a fresh clone. Results record the models, CLI versions, commits and a digest of the agent files, and report pass rates with 95% intervals rather than a verdict. Set the cap with --budget-usd, cedr bench budget or the app's Settings page. Ctrl-C stops the agent and keeps what the run measured. Doctor checks it, MCP gains bench_plan, bench_history and bench_show, and the GitHub Action gains bench, bench-budget-usd, bench-fail-on and bench-allow-unpriced. Bench needs the repo's full history, so set fetch-depth: 0 in CI. A suite is code, so run ones you trust. Contract: docs/bench.md.
  • Guard adds cedr guard lock --remote and cedr guard remote, which report added and removed tools, schema and description changes, new hosts and expanded capabilities. They send no credentials and contact only public HTTPS addresses. A server that is down or needs a login reports as that, never as unchanged. [remote] interval_minutes limits how often a scheduled run really reads a server. Contract: docs/design.md.
  • Route has a built-in grok reader for Grok Build's weekly allowance, which Council's picks use too. cedr monitor limits shows the reading, cedr doctor checks Grok Build's MCP registration, and cedr route import sets its effort option. The plan's pool is shared with the vendor's other products, so the percent can rise with no Grok Build turn. Contract: docs/route.md.
  • Route gives you control of its limits and models. [thresholds] can now be set per plan and per route, session and weekly limits can differ (session_out, weekly_out), and conserve_sizes and unknown_sizes choose what work a low or unread plan still takes. A candidate's sizes sends small and large work to different models. cedr route thresholds, add, remove, enable and disable make these changes for you, keeping your file's comments; cedr route show explains what a route would pick at each size and why, and cedr route models lists what you can choose from. Older cedr versions refuse a routes.toml using the new keys, so upgrade every machine that shares one. Contract: docs/route.md.
  • Monitor adds responseSet to each row it pushes, a digest of the responses behind the row, so a log copied between your machines counts once in the app. Past hours count once after cedr monitor push --full on each machine. The responses' identities never leave the machine. Contract: docs/monitor.md.
  • Versioning is now written down: what counts as public API, what each bump means, and how breaking changes reach you. Read docs/versioning.md.
Full changelog

What's Changed

  • docs(agents): record an approval only when a maintainer gave it for that exact head
  • docs: drop the 0.5.0 caveat from the self-update note
  • docs: stop naming specific versions in examples so they cannot go stale
  • docs(guard): drop the symlink gap that guard already handles (#16)
  • feat(route): read grok build plan usage so picks can balance it (#16)
  • feat(monitor): identify the responses behind each pushed row (#17)
  • feat(guard): detect tool changes on remote mcp servers (#18)
  • fix(guard): report servers that drop tools and keep overlapping reads apart (#19)
  • fix(guard): fail when a remote read cannot be recorded (#20)
  • test(guard): skip the unwritable directory case when running as root (#21)
  • build(banner): render the v0.8.0 release hero
  • docs: changelog for 0.8.0
  • chore(release): cedr 0.8.0
  • docs(agents): let agents choose changelog section order
  • docs: define the version scheme and what it promises
  • feat(bench): measure whether an agent change helps with task suites
  • docs: lead the 0.8.0 changelog and hero with bench
  • fix(bench): stop a run whose spend the budget cannot see
  • feat(route): let users set thresholds and models per plan, route and size
  • fix(bench): refuse to run when the saved budget cannot be read
  • fix(bench): stop the agent on ctrl-c and keep saved runs readable
  • fix(route): keep each comment on its own line when editing routes.toml
  • docs: finish the 0.8.0 changelog after the release review
  • fix(route): keep comments through awkward candidate lists and windows line endings
  • fix(bench): report an interrupt as the stop reason and save the run if output is gone
  • fix(route): name the routes file as given when reporting an edit
  • docs: say how a copied log counts once in the 0.8.0 changelog

Full Changelog: v0.7.0...v0.8.0