fix(routing): per-cluster alpha_floor so the price/quality dial keeps every cluster at its best-for-budget model#501
Merged
Conversation
…t strand agentic on a cheap model The QualityBias dial resolved to a single uniform alpha, overwriting the bundle's shaped per-cluster alpha vector (0.96 on agentic/code clusters, 0.8 on conversational). That shape exists precisely so a price-leaning dial never cheaps out agentic main-loop turns — but flattening it discarded the protection: a mid dial (≈0.60 effective alpha) flips the agentic cluster from the frontier to the cheapest "good-enough" model, which in prod routed a `/cloud-dbg` skill turn to minimax-m3, a model that can't drive the Claude Code tool/skill protocol (it grepped the filesystem for the skill instead of running it). Fix: applyDialAlpha maps the dial through the existing mix-change calibration and then floors only a shaped bundle's top-alpha (most-protected) clusters at dialAlphaFloorFraction of their default; lower-alpha clusters stay fully dial-able. Uniform/legacy bundles (e.g. v0.67) are untouched. Shared by Route and RoutingDistribution so the dashboard preview matches live routing. Verified on the v0.70 corpus via cmd/routing-report --quality-bias (new flag). At quality_bias=0.2: agentic_tool 0%→73% premium, hard_code 0%→57%, while conversational stays ~87% cheap — the dial still saves money where it's safe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Reviews (1): Last reviewed commit: "fix(routing): floor protected clusters s..." | Re-trigger Greptile |
…floor in bundle metadata Replaces the hardcoded floor (a magic 0.92 constant + a heuristic that GUESSED protected clusters from the alpha shape) with an explicit, declarative per-cluster `alpha_floor` vector that the bundle ships in `default_routing_knobs`. The bundle now states, per cluster, the lowest quality weight it tolerates at maximum price-sensitivity; applyDialAlpha just does `alpha[i] = max(dialToAlpha(t), floor[i])`. nil/absent floor = legacy uniform dial (no behavior change for bundles that don't ship it, e.g. v0.67). v0.70 ships a full floor vector: coding/agentic/knowledge clusters at 0.88 (stay on premium models at any dial), conversational/trivial at 0.55 (still far cheaper than their 0.80 default, but not collapsed to the cheapest model — so price-sensitive turns get the best quality their budget allows). Validated at load (length == K, each in [0,1]). Verified on the v0.70 corpus via cmd/routing-report --quality-bias 0.2: agentic_tool 53% opus / 35% glm (minimax gone), conversational best-for-budget (deepseek-v4-pro), climbing to ~all-opus at the quality end. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
🧭 Cluster routing review
Cluster routing report —
|
| register | Opus % | premium % | top models |
|---|---|---|---|
| conversational | 33% | 43% | deepseek-v4-pro:16 claude-opus-4-8:10 gemini-3.1-pro-preview:2 gemini-3.5-flash:1 |
| trivial_nl | 13% | 50% | deepseek-v4-pro:13 gemini-3.1-pro-preview:10 claude-opus-4-8:4 gpt-5.5:1 |
| knowledge_qa | 70% | 77% | claude-opus-4-8:21 glm-5.2:4 deepseek-v4-pro:2 gemini-3.1-pro-preview:1 |
| easy_code | 73% | 87% | claude-opus-4-8:22 deepseek-v4-pro:3 gemini-3.1-pro-preview:2 gpt-5.5:2 |
| hard_code | 60% | 80% | claude-opus-4-8:18 gpt-5.5:6 glm-5.2:6 |
| agentic_tool | 87% | 93% | claude-opus-4-8:26 gpt-5.5:2 glm-5.2:2 |
What each cluster represents (probe register mix by nearest centroid)
| cluster | probes | dominant | register breakdown |
|---|---|---|---|
| 0 | 36 | agentic_tool | agentic_tool:12 knowledge_qa:9 hard_code:8 conversational:4 easy_code:2 trivial_nl:1 |
| 1 | 8 | easy_code | easy_code:3 agentic_tool:2 conversational:1 knowledge_qa:1 trivial_nl:1 |
| 2 | 2 | conversational | conversational:2 |
| 3 | 1 | knowledge_qa | knowledge_qa:1 |
| 4 | 3 | conversational | conversational:1 easy_code:1 trivial_nl:1 |
| 5 | 14 | agentic_tool | agentic_tool:5 conversational:2 easy_code:2 hard_code:2 knowledge_qa:2 trivial_nl:1 |
| 6 | 1 | easy_code | easy_code:1 |
| 7 | 6 | agentic_tool | agentic_tool:2 easy_code:2 hard_code:1 knowledge_qa:1 |
| 8 | 9 | knowledge_qa | knowledge_qa:4 agentic_tool:2 hard_code:2 easy_code:1 |
| 9 | 10 | conversational | conversational:4 trivial_nl:4 easy_code:1 knowledge_qa:1 |
| 10 | 24 | conversational | conversational:12 trivial_nl:7 easy_code:3 knowledge_qa:2 |
| 11 | 4 | trivial_nl | trivial_nl:2 easy_code:1 knowledge_qa:1 |
| 12 | 13 | easy_code | easy_code:9 knowledge_qa:2 agentic_tool:1 trivial_nl:1 |
| 13 | 25 | hard_code | hard_code:17 agentic_tool:6 easy_code:1 knowledge_qa:1 |
| 14 | 7 | knowledge_qa | knowledge_qa:4 easy_code:2 trivial_nl:1 |
| 15 | 17 | trivial_nl | trivial_nl:11 conversational:4 easy_code:1 knowledge_qa:1 |
This was referenced Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
The
QualityBiasdial resolved to a single uniform alpha, overwriting the bundle's shaped per-cluster alpha vector (v0.70: 0.96 on coding/agentic/knowledge, 0.8 on conversational). That shape exists so a price-leaning dial never cheaps out agentic turns — but flattening it discarded the protection. A mid dial maps to ≈0.60 effective alpha; at 0.60 the cheapest "good-enough" model wins every cluster. In prod this routed a/cloud-dbgskill turn to minimax-m3, which can't drive the Claude Code tool/skill protocol (it grepped the filesystem for the skill instead of running it).Fix: declarative per-cluster
alpha_floorThe dial now floors each cluster at a bundle-declared minimum:
alpha_flooris an optional per-cluster vector shipped indefault_routing_knobs— the lowest quality weight the bundle tolerates per cluster at maximum price-sensitivity. It is data, not code: no magic constant, no guessing which clusters to protect.nil/absent ⇒ legacy uniform dial (no behavior change for bundles that don't ship it, e.g. v0.67). Validated at load (length == K, each in [0,1]). Shared byRouteandRoutingDistributionso the dashboard preview matches live routing.v0.70 floor vector
Verification
New
--quality-biasflag oncmd/routing-report. On the v0.70 corpus atquality_bias=0.2:Protected clusters are stable across the low half of the dial (floor pins them), then climb to ~all-opus at the quality end. Conversational stays cheap but never collapses to the bottom model.
Tests
TestApplyDialAlpha_HoldsEachClusterAtItsDeclaredFloor— every cluster pinned at its declared floor at t=0; reaches 1.0 at t=1.TestApplyDialAlpha_NilFloorIsUniformDial— no floor ⇒ legacy uniform dial.TestApplyDialAlpha_AgenticStaysOffCheapModelAtLowDial— agentic centroid moves off the cheap model to a frontier model at a low dial.alpha_floor).Notes / open questions
alpha_flooris added to v0.70'sdefault_routing_knobs(hand-maintained routing config, like the existing shapedalpha). It's additive and changes no clustering/quality data, but it does alter a same-day-shipped deployed version in place — flag if you'd prefer a v0.71 bump.top_p=2(a cluster blends with its nearest neighbor), so an agentic cluster paired with an unprotected neighbor can partially cheapen at the extreme low dial. The reported{0,4}case (both protected) is fully fixed.