Repository navigation
Replies: 1 comment
|
This is already how routing works on main: every cross-vendor lane carries a data-class ceiling (PUBLIC-only for the Google and xAI lanes) and |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
With model routing coming to LifeOS, one suggestion from running a multi-provider setup for a while: route on two dimensions.
Picking the cheapest model that's smart enough is the capability dimension. The second dimension is which providers are allowed to see a given class of data.
In practice I keep a simple ceiling per provider (some vendors only get public-topic work, others get higher sensitivity data) and the router enforces it, so a task carrying personal or client context can never land on a provider I wouldn't hand it to directly.
Capability routing saves tokens. Sensitivity routing is what makes multi-provider safe for real work, especially for anyone running LifeOS with client or company data nearby.
If the router ships with both dimensions from day one, a lot of the compatibility requests in the discussions here (Codex, local models, Hermes) get easier to grant safely.
Longer version of a comment I left on #1681.
All reactions