Support automatically switching provider for similar or same models #9801
charlieadair
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Before submitting:
Area
provider orchestration ?
Problem or use case
I have access to the same model from multiple different places. For example: Claude Fable is available from my max subscription AND provided by GitHub copilot (via Opencode) with my education benefits alongside other Anthropic and Codex models as well. Additionally, OpenRouter has options for all of these.
Proposed solution:
It would be nice if there was a way to recognize that these are all the same model underneath and have a way to route to the next cheapest, or prioritize using THIS provider before THAT provider.
I imagine this would work by first grouping similarly named models between providers > configuring priority > showing it as one model within the UI. When you hit a 402 payment required for the next API request, switch to the next provider. This logic could probably extend to switching similarly capable models (Astra to Fable, for instance).
Why this matters
High potential to save money and utilize free and/or cheaper providers first. Also allows you to automatically switch to a different provider and use the specified model even when the first endpoint is down or busy.
Having multiple providers, particularly in OpenCode also causes confusion on which opencode/anthropic/X I am hitting. Whether its from GitHub copilot, Amazon Bedrock, etc.
Smallest useful scope
Once I have taken advantage of free usage limits within GitHub copilot's Fable endpoint (via OpenCode) and hit a 402 -> auto switch to my claude subscription. Once I am out of fable credits on my claude subscription -> use Amazon Bedrock via openrouter.
Alternatives considered:
From my understanding, #2829 adds the ability to do cross-provider orchestration. This could just be a definition for the agent to use from the specific agent markdown definition. "If this provider gives you problems -> use this one." This isn't a great solution however because it still crowds your model selector with identical names.
Risks or tradeoffs
In some situations, taking advantage of prompt cache-ing is going to be cheaper than switching to a different provider.
Examples or references
Idk what this means, sorry.
Contribution
I would love to get the opportunity to contribute to an open source project like this. Thanks.
All reactions