Skip to content

v0.23.0: OpenCode, Antigravity, and hillclimb-skill

Choose a tag to compare

@edonadei edonadei released this 15 Nov 04:13
452aaf4

Caliper is going well, and I'm getting more confident in the software factory behind it. That confidence let me add two new backends in one release: OpenCode and Antigravity.

New backends

  • OpenCode: run evals with --model opencode or --model opencode:<provider/model>. Supports skill discovery, MCP servers, user customizations, planted conversation history, token usage, and judging. Sign in once with opencode auth login. Setup and details. (#345)
  • Antigravity: run evals through Google's agy CLI with --model antigravity or --model agy. Supports skills, MCP servers, user customizations, token usage, and judging. Planted conversation history is not supported yet. Setup and details. (#344)

Both backends run attempts in isolated environments and disable automatic CLI updates during evals.

New skill

  • hillclimb-skill improves an existing skill against its eval, one change per round. It separates training tasks from held-out tasks, measures baseline noise, and keeps changes only when they meet its improvement criteria on both splits. You can aim for a higher score or fewer tokens at the same score. Start with /hillclimb-skill ./your-skill/SKILL.md; it needs an eval with at least 8 tasks. (#343)

Docs

  • The README quick-start paths are stacked for easier reading. (#342)

Upgrade

pip install --upgrade caliper-eval

Full changelog: v0.22.1...v0.23.0