Skip to content

v1.56.2

Choose a tag to compare

@github-actions github-actions released this 09 Nov 23:10
· 304 commits to main since this release

Haiku only for a quick question the local model is sure about

Model per message now sends a message to Haiku only when all of these are true:

  • the bge-small vote is for Haiku at 70% or more;
  • the message looks like a known example;
  • it is short (20 words or fewer);
  • it asks for nothing that sounds like work: making or changing something, a status check, a "why", something not showing up, a file path or a link.

Anything else runs on Sonnet. The word rules alone never pick Haiku, and a sure vote now overrides any rule that disagrees.

Measured on 3,000 real prompts: heavy turns sent to Haiku (15+ tool calls or 12k+ output tokens) fell from 167 (11.4% of heavy turns in v1.56.1) to 2 (0.1%). Decision time is unchanged (about 30 ms at the median).

For contributors

npm run router:eval measures the router on your own prompts. It runs outside the test suite and coverage, and nothing leaves the computer.

  • It scores the 120 seed examples against each other.
  • It replays up to 3,000 of your recent typed Claude Code prompts and compares each pick with how much work answering actually took.
  • It writes a 300-prompt labelling sheet. Fill it in and re-run to get true accuracy.

Full Changelog: v1.56.1...v1.56.2