Skip to content

feat(review): per-agent model + effort, defaulting to a cheaper review - #12

Merged
craigmbooth merged 1 commit into
mainfrom
feat/review-per-agent-model-and-effort
Aug 1, 2026
Merged

feat(review): per-agent model + effort, defaulting to a cheaper review#12
craigmbooth merged 1 commit into
mainfrom
feat/review-per-agent-model-and-effort

Conversation

@craigmbooth

Copy link
Copy Markdown
Collaborator

Stacked on #11 — base is fix/gates-enforce-exit-codes-and-skip-unchanged-deps, not main. Merge #11 first; the base will retarget automatically.

What

The review workflow ran both agents on a single model at the model's own default effort. This splits that into per-agent controls and moves two defaults down, so consumers get a cheaper review on the bump without editing anything.

Why, with numbers

Measured on a real intendent PR (the agents self-report total_cost_usd):

Agent Turns Cost
review 18 $1.00
test-plan 14 $0.53

$1.54 per pull request. The two halves aren't equally hard: the reviewer does adversarial audit, where model tier buys real findings; the test-plan agent does read-only analysis and emits a checklist — well inside Haiku's range, at $1/$5 per MTok against Sonnet's $3/$15.

Changed defaults

  • test-plan-modelclaude-haiku-4-5 (was: inherited model, i.e. Sonnet 4.6)
  • review-effortmedium (was: the model's own high)

The reviewer deliberately stays on Sonnet. Dropping it to Haiku would save more and cost exactly the thing this workflow exists to produce.

New inputs

review-model, test-plan-model, review-effort, test-plan-effort. Empty model falls back to model; empty effort passes no flag at all, so an unconfigured agent behaves exactly as before.

Two sharp edges, both documented

effort is rejected by Haiku 4.5 and Sonnet 4.5. That's why test-plan-effort defaults to empty — by necessity, not preference. Setting it while test-plan-model is on the default Haiku will fail the request. This is the one thing most likely to bite someone; it's called out at the input, in the README, and in the upgrade notes.

model: no longer reaches both agents. test-plan-model's non-empty default takes precedence over the fallback, so a caller setting model: and expecting it everywhere now needs a matching test-plan-model:.

For the reviewer

  • Changelog placement. Folded into the in-flight v3.0.0 rather than claiming its own version — this stacks on fix(gates)!: tee no longer masks exit codes; skip deps scan when unchanged #11 and changes default behavior, which is MAJOR under LIFECYCLE regardless. If you'd rather it be a separate release, it needs splitting out before tag.
  • I edited fix(gates)!: tee no longer masks exit codes; skip deps scan when unchanged #11's upgrade-notes prose. Its "no gate default flipped, so the bump is mechanical" line became inaccurate once review defaults moved, so I scoped that claim to the gate half and added the review-side note.
  • Untested against a live run. review.yml YAML-validates and the expressions render correctly, but no PR has actually executed with these defaults. First real exercise will be this PR's own review — which, being a workflow-file change, may trip the GitHub App workflow-validation guard and skip.

Also worth knowing

v2.4.0 has a changelog entry on main but no git tag — the review-ticket skill is merged and unreleased. And #11 cuts v3.0.0 without bumping plugin/.claude-plugin/plugin.json, which still reads 2.4.0; past releases kept it in lockstep with the single SemVer stream. Both are outside this PR's scope but want resolving before the tag.

🤖 Generated with Claude Code

https://claude.ai/code/session_01T3dRiaun8tKHmqQWCpMWMo

The review workflow ran both agents on one model at the model's own default
effort. Measured on a real intendent PR that is $1.54 per pull request:
$1.00 for the reviewer (18 turns) and $0.53 for the test-plan agent (14).

The two halves are not equally hard. The reviewer does adversarial audit,
where model tier buys real findings. The test-plan agent does read-only
analysis and emits a checklist — work well inside Haiku's range, at $1/$5
per MTok against Sonnet's $3/$15. Splitting the single `model` input lets
each sit where it belongs.

Adds four inputs — review-model, test-plan-model, review-effort,
test-plan-effort — and changes two defaults so consumers get the saving on
the bump without editing anything: test-plan-model to claude-haiku-4-5, and
review-effort to medium (below the model's own high). An empty model falls
back to `model`; an empty effort passes no flag at all, so any agent left
unconfigured behaves exactly as before.

test-plan-effort defaults to empty by necessity: effort is rejected by
Haiku 4.5 and Sonnet 4.5, so pairing it with the default test-plan-model
would fail the request. Documented at the input, in the README, and in the
upgrade notes.

Note for callers that set `model:` — it no longer reaches both agents, since
test-plan-model's non-empty default takes precedence over the fallback.

Changelog lands under the in-flight v3.0.0 rather than claiming a release of
its own: this stacks on the gates-exit-code fix and changes default
behavior, which is MAJOR under LIFECYCLE either way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T3dRiaun8tKHmqQWCpMWMo
@craigmbooth
craigmbooth changed the base branch from fix/gates-enforce-exit-codes-and-skip-unchanged-deps to main August 1, 2026 23:08
@craigmbooth
craigmbooth merged commit 59b8dac into main Aug 1, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant