You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Pace-aware model selection: spread usage across a limit window and keep a buffer for the last 24h
#37305
On subscription plans, any usage left over when a weekly window resets is lost. Usage that runs out early is lost too, because it means waiting days for the reset. T3 already has most of the pieces to avoid both, but nothing connects them:
fallbackModelSelection is only used when a new thread starts on a default that's already exhausted (resolveNewThreadModelSelection). Until then, new work keeps landing on a provider that's far ahead of pace.
So with several providers connected, I either run one into the wall mid-task or end the week with 40% of another unused.
Proposal
An opt-in pace-aware selection for new work, based on a target curve for each provider's usage window.
Target curve (weekly window):
reserve = 24h (configurable), buffer = 10% (configurable)
before the reserve: target = (100% − buffer) × elapsed / (period − reserve)
during the reserve: target rises from (100% − buffer) to 100% by the reset
So usage is spread evenly over the week, a 10% buffer is held back for the last 24 hours, and that buffer is used up before the reset.
Short windows (e.g. the 5-hour window): a simple ceiling, for example 85%, instead of a curve.
Behaviour:
New threads, queued runs and delegated tasks go to the highest-ranked model in the user's fallback order whose provider is at or under its target in every window. If none qualifies, they go to the model furthest under its target.
Providers that report no usage windows count as always under pace, or can be excluded.
The routing decision and its reason are shown, e.g. "Started on Codex: Claude is 12% ahead of its weekly pace."
First slice
Read-only first: show the target, and how far ahead or behind it each provider is, next to the existing pace tick. Then use it in resolveNewThreadModelSelection behind a setting that's off by default, alongside the existing fallback.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Problem
On subscription plans, any usage left over when a weekly window resets is lost. Usage that runs out early is lost too, because it means waiting days for the reset. T3 already has most of the pieces to avoid both, but nothing connects them:
fallbackModelSelectionis only used when a new thread starts on a default that's already exhausted (resolveNewThreadModelSelection). Until then, new work keeps landing on a provider that's far ahead of pace.autoResumeLimitedThreadsand the fallback handoff in Option to automatically hand off to a fallback provider when hitting usage limits #15846 handle a thread that has already hit a limit. They don't prevent hitting it.So with several providers connected, I either run one into the wall mid-task or end the week with 40% of another unused.
Proposal
An opt-in pace-aware selection for new work, based on a target curve for each provider's usage window.
Target curve (weekly window):
So usage is spread evenly over the week, a 10% buffer is held back for the last 24 hours, and that buffer is used up before the reset.
Short windows (e.g. the 5-hour window): a simple ceiling, for example 85%, instead of a curve.
Behaviour:
First slice
Read-only first: show the target, and how far ahead or behind it each provider is, next to the existing pace tick. Then use it in
resolveNewThreadModelSelectionbehind a setting that's off by default, alongside the existing fallback.Related
I'm prototyping this locally with a scheduled task that reads each provider's usage windows and picks the model for new work.
Drafted with Claude Code.
All reactions