Repository navigation
v0.2.9
Reasoning effort is now a dial you can turn
One canonical ladder — low < medium < high < xhigh < max — mapped to whatever each backend's CLI calls it, in one auditable place. A rung a backend cannot express is clamped down, never up, and the clamped value is what gets recorded, so telemetry says what actually ran rather than what was asked for.
agentpit rescue --backend claude --effort xhigh "…"
agentpit ensemble --effort high "…"
agentpit profile run --backend codex --effort xhighAlso configurable as [backends.<id>].effort and [workflow.roles.<name>].effort, and exposed on the MCP dispatch_task / run_ensemble tools. Precedence mirrors model exactly: flag > role > backend default.
| canonical | claude | codex | agy | opencode |
|---|---|---|---|---|
| low / medium / high | same | same | same | same |
| xhigh | xhigh |
xhigh |
clamped to high |
xhigh |
| max | max |
clamped to xhigh |
clamped to high |
max |
agy binds effort to the model id (gemini-3.6-flash-high) and rejects a separate --effort, so a pinned model wins and says so instead of failing the dispatch.
Capability rows are keyed by what produced them
A capability score was never a property of a CLI's name — it belongs to (backend, model, effort). profiles.toml now keys on the triple, so a high-effort measurement and a low-effort one coexist instead of one silently overwriting the other. An unmeasured variant falls back to the unpinned row and never borrows a sibling rung's number.
Existing files load unchanged: a bare [profile.codex] is the unpinned row it always was.
agentpit arena — blind head-to-head, judged by a human
agentpit arena run --contenders claude,codex "<what to build>"
agentpit arena vote
agentpit arena leaderboardEvery contender builds the same thing in its own git worktree; its work is reduced to a patch; you pick the better one blind and pairwise, with the names withheld until every vote in the round is in. Standings come from a Bradley–Terry fit with a bootstrap interval — a unanimous handful of votes is marked provisional rather than presented as a finding. Votes flow into the ordinary learning fold as MemberGraded, not as a new source that outranks a measured benchmark.
Two long-standing bugs, found by running the real CLIs
Both had passing unit tests the whole time; the tests asserted argument construction, which is not the same as the flag working.
--modelnever reached antigravity.agy --printis an alias for--promptand eats the next token as its value, so any flag after it was swallowed into the prompt.- codex could not write files.
codex execsandboxes to read-only by default and the adapter set none, so codex dispatches described the edit they would have made and changed nothing — while declaring full autonomy. Now--sandbox workspace-write;danger-full-accessstays reserved for the workflow manager.