Repository navigation
v0.23.0: OpenCode, Antigravity, and hillclimb-skill
Caliper is going well, and I'm getting more confident in the software factory behind it. That confidence let me add two new backends in one release: OpenCode and Antigravity.
New backends
- OpenCode: run evals with
--model opencodeor--model opencode:<provider/model>. Supports skill discovery, MCP servers, user customizations, planted conversation history, token usage, and judging. Sign in once withopencode auth login. Setup and details. (#345) - Antigravity: run evals through Google's
agyCLI with--model antigravityor--model agy. Supports skills, MCP servers, user customizations, token usage, and judging. Planted conversation history is not supported yet. Setup and details. (#344)
Both backends run attempts in isolated environments and disable automatic CLI updates during evals.
New skill
hillclimb-skillimproves an existing skill against its eval, one change per round. It separates training tasks from held-out tasks, measures baseline noise, and keeps changes only when they meet its improvement criteria on both splits. You can aim for a higher score or fewer tokens at the same score. Start with/hillclimb-skill ./your-skill/SKILL.md; it needs an eval with at least 8 tasks. (#343)
Docs
- The README quick-start paths are stacked for easier reading. (#342)
Upgrade
pip install --upgrade caliper-evalFull changelog: v0.22.1...v0.23.0