💡 Claude Sonnet 5 Engine Chain Integration with Introductory Pricing Capture

[github-actions[bot]]

Summary

Evaluate and integrate Claude Sonnet 5 (model ID: claude-sonnet-5-*) into engine.sh model chains as a potential Sonnet 4.6 replacement. Sonnet 5 launched June 30, 2026 with introductory pricing at $2/$10 per MTok I/O (33% cheaper than Sonnet 4.6 on input) through August 31, and scores 63.2% on SWE-bench Pro vs Sonnet 4.6's 58.1% — an 8.8% coding quality improvement at lower cost.

Market Signal

Claude Sonnet 5 launched June 30, 2026 (anthropic.com/news/claude-sonnet-5). Key data points:

• Introductory pricing (through Aug 31): $2/MTok input, $10/MTok output
• Standard pricing (after Aug 31): $3/MTok input, $15/MTok output
• SWE-bench Pro: 63.2% (vs Sonnet 4.6 at 58.1%, Opus 4.8 at 69.2%)
• Terminal-Bench 2.1: 80.4% (vs Sonnet 4.6 at 67.0%)
• Humanity's Last Exam (tools): 57.4% (near Opus 4.8 at 57.9%)
• Rate limit equalization: Sonnet/Haiku now match Opus limits at every usage tier — changes the model-chain fallback calculus (engine.sh walks sonnet→opus on rate-limit)
• Refusal billing change: API requests returning stop_reason: "refusal" are no longer billed — impacts token cost reporting accuracy

The pricing war is intensifying: Google cut AI Ultra plan 20%, Chinese alternatives (DeepSeek, Kimi) undercut by up to 9x, and Anthropic shifted enterprise contracts to usage-based billing. Cost-performance optimization is increasingly table stakes.

User Signal

• engine.sh already has CLAUDE_*_MODEL_CHAIN infrastructure for in-provider model fallback
• model-pricing.tsv supports effective-dated pricing rows — add introductory and standard rate rows
• The canary-ring model (canary-rollout.sh, cut-release.sh) provides a proven evaluation/promotion path
• The Opus 4.8 evaluation initiative (Opus 4.8 engine upgrade: validate the deep-review tier, finish the reference rollout, and keep cost data accurate #835) is in flight — Sonnet 5 is a separate evaluation at a different tier
• Issues dev-lead emits empty, model-less token records that pollute the "no price" count in the cost report #1009 (empty token records), [dev-lead timeout follow-up 3/3] Report timeout wall-rate as prevalence (over total dev-lead runs) #1030 (timeout wall-rate) would benefit from a more capable default Sonnet tier
• Rate limit equalization may allow simplifying the sonnet→opus fallback chain

Technical Opportunity

The integration path is well-established:

1. Pricing data: Add claude-sonnet-5-* rows to model-pricing.tsv with 2026-06-30 effective date at $2/$10 and 2026-09-01 at $3/$15
2. Model chain: Wire claude-sonnet-5 into CLAUDE_TRIAGE_MODEL_CHAIN and/or CLAUDE_DEEP_MODEL_CHAIN as a candidate
3. Canary evaluation: Promote through next → ring0 → ring1 → stable with the eval suite (💡 SkillOpt-Style Self-Improving Skills: Eval-Gated, Human-Reviewed Agent Skill Evolution #572, Skill eval health — triage (holdout) #998) as quality gate
4. Fallback simplification: Evaluate whether rate-limit equalization makes the sonnet→opus in-engine fallback unnecessary for RPM/TPM scenarios (daily subscription cap fallback remains needed)

Assessment

Dimension	Score	Rationale
Feasibility	high	Infrastructure exists: pricing rows, model chains, canary rings. Configuration change + evaluation, not new architecture.
Impact	high	33% input cost reduction + 8.8% SWE-bench quality improvement affects every agent session fleet-wide.
Urgency	high	Introductory pricing window closes Aug 31. Capturing it requires evaluation + canary promotion within ~8 weeks.

Adversarial Review

Strongest objection: This is routine model maintenance, not a feature idea — every Claude API consumer will do this evaluation.

Rebuttal: The model chain routing is the core decision layer of the agent harness — which model runs where determines both cost and quality for every agent session fleet-wide. A model that is simultaneously 33% cheaper and 8.8% better at coding tasks is not routine — it is a material shift in the cost-quality frontier. The introductory pricing window creates genuine urgency: the project needs to evaluate, canary-promote, and capture the $2/$10 rate before August 31. And the rate-limit equalization is a structural change to the fallback calculus, not just a price drop.

Suggested Next Step

Add claude-sonnet-5-* pricing rows to model-pricing.tsv (two rows: introductory effective 2026-06-30, standard effective 2026-09-01), wire into the triage-tier model chain as a candidate, and evaluate through canary rings with the existing eval suite as the quality gate. Prioritize to capture the introductory pricing window.

[github-actions[bot]]

📋 Initiative planned by the BMAD Scrum Master (Bob).

Epic #1095 — Initiative: Integrate Claude Sonnet 5 into engine.sh model chains — capture introductory pricing, eval-gate quality, canary-promote

5 stories created (inert — labelled initiative, NOT initiative:auto):

• [Phase 1] Add effective-dated claude-sonnet-5 pricing rows + lock them with a regression test #1096 (S) — [Phase 1] Add effective-dated claude-sonnet-5 pricing rows + lock them with a regression test
• [Phase 1] Held-out eval regression gate: Sonnet 5 >= Sonnet 4.6 on triage + deep-review #1097 (M) — [Phase 1] Held-out eval regression gate: Sonnet 5 >= Sonnet 4.6 on triage + deep-review
• [Phase 2] Wire claude-sonnet-5 into the Claude model chains as a candidate #1098 (M) — [Phase 2] Wire claude-sonnet-5 into the Claude model chains as a candidate
• [Phase 2] Assess simplifying the sonnet->opus in-engine fallback given rate-limit equalization #1099 (M) — [Phase 2] Assess simplifying the sonnet->opus in-engine fallback given rate-limit equalization
• [Phase 3] Canary-promote via the dev-lead rings and record the human go/no-go #1100 (M) — [Phase 3] Canary-promote via the dev-lead rings and record the human go/no-go

Open questions for review:

• Refusal-billing accuracy (Discussion 💡 Claude Sonnet 5 Engine Chain Integration with Introductory Pricing Capture #1039 data point): Anthropic no longer bills requests returning stop_reason: "refusal". The estimate-based token-metrics.sh does not observe refusals, so cost reports could slightly over-report Sonnet 5 refusal turns. This is out of scope for the model swap itself — confirm whether to open a separate follow-up rather than fold it into this initiative.
• Tier priority: the idea says wire Sonnet 5 into the triage AND/OR deep chain. Stories 2-3 defer the specific tier to the eval outcome, but confirm whether there is a preferred first target (triage vs deep) if the eval is inconclusive or a phased tier-by-tier rollout is wanted.

---

Review the epic and its sub-issue DAG, adjust as needed, then add initiative:auto to epic #1095 to hand it to initiative-driver for auto-implementation.