Skip to content

fix(routing): per-cluster alpha_floor so the price/quality dial keeps every cluster at its best-for-budget model#501

Merged
steventohme merged 2 commits into
mainfrom
steven/dial-per-cluster-alpha-floor
Jun 24, 2026
Merged

fix(routing): per-cluster alpha_floor so the price/quality dial keeps every cluster at its best-for-budget model#501
steventohme merged 2 commits into
mainfrom
steven/dial-per-cluster-alpha-floor

Conversation

@steventohme

@steventohme steventohme commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Root cause

The QualityBias dial resolved to a single uniform alpha, overwriting the bundle's shaped per-cluster alpha vector (v0.70: 0.96 on coding/agentic/knowledge, 0.8 on conversational). That shape exists so a price-leaning dial never cheaps out agentic turns — but flattening it discarded the protection. A mid dial maps to ≈0.60 effective alpha; at 0.60 the cheapest "good-enough" model wins every cluster. In prod this routed a /cloud-dbg skill turn to minimax-m3, which can't drive the Claude Code tool/skill protocol (it grepped the filesystem for the skill instead of running it).

Fix: declarative per-cluster alpha_floor

The dial now floors each cluster at a bundle-declared minimum:

alpha[i] = max(dialToAlpha(t), floor[i])

alpha_floor is an optional per-cluster vector shipped in default_routing_knobs — the lowest quality weight the bundle tolerates per cluster at maximum price-sensitivity. It is data, not code: no magic constant, no guessing which clusters to protect. nil/absent ⇒ legacy uniform dial (no behavior change for bundles that don't ship it, e.g. v0.67). Validated at load (length == K, each in [0,1]). Shared by Route and RoutingDistribution so the dashboard preview matches live routing.

Supersedes the first revision of this PR, which hardcoded a 0.92 constant + inferred protected clusters from the alpha shape. That coupled routing policy to scorer code and was fit to one corpus.

v0.70 floor vector

  • Coding / agentic / knowledge clusters → 0.88 (stay on premium models at any dial)
  • Conversational / trivial clusters → 0.55 (still far cheaper than their 0.80 default, but not collapsed to the cheapest model — best quality for their budget)

Verification

New --quality-bias flag on cmd/routing-report. On the v0.70 corpus at quality_bias=0.2:

register before (uniform dial) after (alpha_floor)
agentic_tool 0% premium → cheapest (minimax/flash) 53% opus + 35% glm-5.2 (no minimax)
hard_code 0% premium 40% opus / 57% glm-5.2
knowledge_qa low 53% opus
conversational cheapest model deepseek-v4-pro / mimo (best-for-budget)

Protected clusters are stable across the low half of the dial (floor pins them), then climb to ~all-opus at the quality end. Conversational stays cheap but never collapses to the bottom model.

Tests

  • TestApplyDialAlpha_HoldsEachClusterAtItsDeclaredFloor — every cluster pinned at its declared floor at t=0; reaches 1.0 at t=1.
  • TestApplyDialAlpha_NilFloorIsUniformDial — no floor ⇒ legacy uniform dial.
  • TestApplyDialAlpha_AgenticStaysOffCheapModelAtLowDial — agentic centroid moves off the cheap model to a frontier model at a low dial.
  • Existing v0.67 distribution/dial tests unchanged (ships no alpha_floor).

Notes / open questions

  • In-place edit of v0.70 metadata. alpha_floor is added to v0.70's default_routing_knobs (hand-maintained routing config, like the existing shaped alpha). It's additive and changes no clustering/quality data, but it does alter a same-day-shipped deployed version in place — flag if you'd prefer a v0.71 bump.
  • Known residual: prod runs top_p=2 (a cluster blends with its nearest neighbor), so an agentic cluster paired with an unprotected neighbor can partially cheapen at the extreme low dial. The reported {0,4} case (both protected) is fully fixed.
  • Floor values are now trivially tunable in metadata — raise the 0.88 entries if you want agentic even more opus-heavy at a price dial.

…t strand agentic on a cheap model

The QualityBias dial resolved to a single uniform alpha, overwriting the
bundle's shaped per-cluster alpha vector (0.96 on agentic/code clusters, 0.8
on conversational). That shape exists precisely so a price-leaning dial never
cheaps out agentic main-loop turns — but flattening it discarded the
protection: a mid dial (≈0.60 effective alpha) flips the agentic cluster from
the frontier to the cheapest "good-enough" model, which in prod routed a
`/cloud-dbg` skill turn to minimax-m3, a model that can't drive the Claude Code
tool/skill protocol (it grepped the filesystem for the skill instead of running
it).

Fix: applyDialAlpha maps the dial through the existing mix-change calibration
and then floors only a shaped bundle's top-alpha (most-protected) clusters at
dialAlphaFloorFraction of their default; lower-alpha clusters stay fully
dial-able. Uniform/legacy bundles (e.g. v0.67) are untouched. Shared by Route
and RoutingDistribution so the dashboard preview matches live routing.

Verified on the v0.70 corpus via cmd/routing-report --quality-bias (new flag).
At quality_bias=0.2: agentic_tool 0%→73% premium, hard_code 0%→57%, while
conversational stays ~87% cheap — the dial still saves money where it's safe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jun 24, 2026

Copy link
Copy Markdown

Reviews (1): Last reviewed commit: "fix(routing): floor protected clusters s..." | Re-trigger Greptile

…floor in bundle metadata

Replaces the hardcoded floor (a magic 0.92 constant + a heuristic that GUESSED
protected clusters from the alpha shape) with an explicit, declarative
per-cluster `alpha_floor` vector that the bundle ships in
`default_routing_knobs`. The bundle now states, per cluster, the lowest quality
weight it tolerates at maximum price-sensitivity; applyDialAlpha just does
`alpha[i] = max(dialToAlpha(t), floor[i])`. nil/absent floor = legacy uniform
dial (no behavior change for bundles that don't ship it, e.g. v0.67).

v0.70 ships a full floor vector: coding/agentic/knowledge clusters at 0.88
(stay on premium models at any dial), conversational/trivial at 0.55 (still far
cheaper than their 0.80 default, but not collapsed to the cheapest model — so
price-sensitive turns get the best quality their budget allows).

Validated at load (length == K, each in [0,1]). Verified on the v0.70 corpus
via cmd/routing-report --quality-bias 0.2: agentic_tool 53% opus / 35% glm
(minimax gone), conversational best-for-budget (deepseek-v4-pro), climbing to
~all-opus at the quality end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@steventohme steventohme changed the title fix(routing): floor protected clusters so the price/quality dial can't strand agentic on a cheap model fix(routing): per-cluster alpha_floor so the price/quality dial keeps every cluster at its best-for-budget model Jun 24, 2026
@github-actions

Copy link
Copy Markdown

🧭 Cluster routing review

ℹ️ Informational — candidate artifact (latest unchanged); no sign-off required.

Cluster routing report — v0.70

Routed 180 labeled probes through the real Scorer at top_p=2, embedder jina-v2-base-code-int8. Premium = opus-4-7/4-8, gpt-5.5, gemini-3.1-pro, sonnet-4-6.

Register → model (this artifact)

register Opus % premium % top models
conversational 33% 43% deepseek-v4-pro:16 claude-opus-4-8:10 gemini-3.1-pro-preview:2 gemini-3.5-flash:1
trivial_nl 13% 50% deepseek-v4-pro:13 gemini-3.1-pro-preview:10 claude-opus-4-8:4 gpt-5.5:1
knowledge_qa 70% 77% claude-opus-4-8:21 glm-5.2:4 deepseek-v4-pro:2 gemini-3.1-pro-preview:1
easy_code 73% 87% claude-opus-4-8:22 deepseek-v4-pro:3 gemini-3.1-pro-preview:2 gpt-5.5:2
hard_code 60% 80% claude-opus-4-8:18 gpt-5.5:6 glm-5.2:6
agentic_tool 87% 93% claude-opus-4-8:26 gpt-5.5:2 glm-5.2:2

What each cluster represents (probe register mix by nearest centroid)

cluster probes dominant register breakdown
0 36 agentic_tool agentic_tool:12 knowledge_qa:9 hard_code:8 conversational:4 easy_code:2 trivial_nl:1
1 8 easy_code easy_code:3 agentic_tool:2 conversational:1 knowledge_qa:1 trivial_nl:1
2 2 conversational conversational:2
3 1 knowledge_qa knowledge_qa:1
4 3 conversational conversational:1 easy_code:1 trivial_nl:1
5 14 agentic_tool agentic_tool:5 conversational:2 easy_code:2 hard_code:2 knowledge_qa:2 trivial_nl:1
6 1 easy_code easy_code:1
7 6 agentic_tool agentic_tool:2 easy_code:2 hard_code:1 knowledge_qa:1
8 9 knowledge_qa knowledge_qa:4 agentic_tool:2 hard_code:2 easy_code:1
9 10 conversational conversational:4 trivial_nl:4 easy_code:1 knowledge_qa:1
10 24 conversational conversational:12 trivial_nl:7 easy_code:3 knowledge_qa:2
11 4 trivial_nl trivial_nl:2 easy_code:1 knowledge_qa:1
12 13 easy_code easy_code:9 knowledge_qa:2 agentic_tool:1 trivial_nl:1
13 25 hard_code hard_code:17 agentic_tool:6 easy_code:1 knowledge_qa:1
14 7 knowledge_qa knowledge_qa:4 easy_code:2 trivial_nl:1
15 17 trivial_nl trivial_nl:11 conversational:4 easy_code:1 knowledge_qa:1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant