Skip to content

v0.16.0 — Claude 5.x compatibility, default model claude-sonnet-5-5

Latest

Choose a tag to compare

@Rynaro Rynaro released this 17 Oct 21:35
04cf227

Since v0.15.0, the Anthropic provider supports the Claude 5.x model family — Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1. Deployments that do not set a Claude model change behavior: the previous default, claude-3-5-sonnet-latest, points to a retired model and was already failing every review with a 404. They now run claude-sonnet-5-5. Set the new ANTHROPIC_MODEL env var to choose another model deployment-wide.

Claude 5.x support (#45)

Model Tool choice Default output cap Input window Token estimate
claude-haiku-5-5, claude-sonnet-5-5, claude-opus-5-5, claude-fable-5-1 (and other 5.x) auto + strict schema + one retry 16,384 1M anthropic-approx-v2 (×1.55)
Claude 4.6 – 4.8 (e.g. claude-sonnet-4-6, claude-opus-4-8) forced (unchanged) 4096 1M Opus 4.7/4.8: anthropic-approx-v2; others: anthropic-approx (×1.15)
Older Claude (e.g. claude-haiku-4-5) forced (unchanged) 4096 200,000 anthropic-approx (×1.15)
  • Opus 5.5, Sonnet 5.5 and Fable 5.1 reject a forced tool call, and Haiku 5.5 skips its thinking step when forced. For 5.x models the reviewer now uses tool_choice: auto with a strict, Anthropic-only copy of the tool schema, and retries once if no tool call comes back. Pre-5 models send exactly the same request as before. ANTHROPIC_TOOL_CHOICE=auto|forced|unforced overrides the rule.
  • Thinking is on by default on 5.x and counts toward max_tokens, so the default cap is 16,384 there. An explicit ANTHROPIC_MAX_OUTPUT_TOKENS always wins.
  • No thinking config and no default effort are sent, so each model uses its own default. A repo can set provider_options.anthropic.effort.

Model selection works

  • New ANTHROPIC_MODEL env var.
  • A per-repo model: anthropic/<id> in .github/review-bot.yml is now honored; the adapter used to drop it silently. So are generation.max_output_tokens and provider_options.anthropic, the latter behind a denylist.
  • temperature, top_p and seed are never sent to Anthropic, because 5.x models reject them.

Failures you can act on

  • Refusals: a safety-classifier refusal (stop_reason: refusal) now gets its own notice and is never retried. Only the declined batch is dropped, and its files get their own "not reviewed" note. Server-side refusal fallbacks are deferred (see ADR-009).
  • Retired models: a 404 now maps to a capability error with a remedy naming current models. It used to be retried as a transport error and never reported.
  • Rejected requests: every 400 invalid_request_error maps to capability. Anthropic-specific remedies cover forced tool choice and thinking/effort rejections, in both the check-run summary and the comment reply.
  • Empty reviews: Claude adaptive-thinking models get their own "no findings" notice.

Observability

  • provider.usage is emitted for Anthropic (endpoint: messages, including cache reads and writes).
  • worker.provider.selected carries model.
  • Refusals carry code on provider.error / provider.batch.error.

Live check

scripts/smoke/anthropic-models.ts runs one real review per model with your own key. It costs a few cents, is never run in CI, and never prints prompts, responses or secrets:

read -rs ANTHROPIC_API_KEY && export ANTHROPIC_API_KEY
./node_modules/.bin/tsx scripts/smoke/anthropic-models.ts

Tests: 1312 across 58 files, all passing; 26/26 deterministic eval scenarios. Details: docs/model-compatibility.md § Anthropic and ADR-009.