NEW FEATURE IDEA: Model Groups/Profiles/Tags

[seanford]

@diegosouzapw Wanted to wait until the new release before I reached out to ask this, but I have a question for you, I am trying to run all of my apps that use AI through Omniroute. As I've set up my Homelab, I've assigned API Keys to each app so I can restrict which models they have access to, etc. However, when a new model comes out, I often have to go back and edit many keys, making the same change in each one. After poking around today, I did see that I can limit keys to providers, and I've always got combos that could do what I'm thinking...to some degree. But I have been thinking about something on my personal wish list for a while and could put some effort into it, but it is a bit of a project, so I wanted to get your feedback on it first.

The idea is model groups or model profiles, and the idea is simple. I'd like to be able to create a Model Group or Profile once, then assign that to an API key, or in my case, many API Keys. That way, when I need to make a change, I only have to change the group or profile and each key that it is assigned to is updated at the same time. One edit, many changes. Right now, I'm asking an AI agent to make these edits in bulk, but I can't help but think there's a better way.

An alternate idea for this (which I admittedly just thought of now), would be model tags. Same concept, but instead of groups, you can assign tags to an API key, and then it would extrapolate a tag into the models tagged with it. Same concept, different structure (and maybe easier to implement).

I was hoping to get your thoughts before I embark on what could be quite a bit of work, without know what the maintainer thinks about the concept, let alone the code.

[SeanFromSoCal]

Isn't that what Combos are for. You can create your own model groups as a combo and then assign that combo to your API key and nothing else. Instruct the apps to use that combo as the model. You can create a combo with all your frontier models. Another one with the better free models available. It has a lot of failover and rotation strategies. You only then need to modify the combos (your model and provider grouping).

[seanford]

Yes and no. A Combo acts like a single model and handles routing, fallback, rotation, etc. in the background. What I’m trying—perhaps poorly—to describe is an access-control layer one level above that.

The application would still request whichever model or Combo it needs. This feature would control which models, provider connections, and Combos its API key is permitted to use.

Because I have LLM accounts all over the place, along with local models for specific purposes such as embeddings, reranking, speech, and image generation, I assign everything that connects to OmniRoute its own API key. I do this for tracking, but also to limit the blast radius if an application or key is compromised.

Not every API key gets access to everything. Right now, I use the model picker and the other key permissions to decide what each key can access. When a new model comes out—or when I add a provider I’m evaluating—I may have to make the same change separately across dozens of keys.

The idea is to create a reusable access definition and assign that to multiple API keys. Providers/connections, models, and Combos could all be members of it. If I add a model to that definition later, every key attached to it receives the updated access automatically.

For example, suppose 20 application keys use a “General AI” access definition. I could add a new model to “General AI” once, and all 20 keys would gain access to it. My embedding, speech, and restricted-production keys would remain unchanged.

I can think of at least three ways to structure it:

1. Profiles — A Profile would be a complete, reusable access configuration. For example, I might create “General AI,” “Coding,” “Local Only,” and “Media Generation” profiles. Each API key could be assigned a Profile, and changes to that Profile would propagate to every attached key. This is probably the simplest mental model and could eventually include other settings such as rate limits or budgets.
2. Groups — A Group would be a modular collection of models, connections, and Combos, with each API key able to join multiple Groups. For example, one key might receive both “General Chat” and “Local Embeddings,” while another receives “General Chat” and “Speech.” This is closest to the Linux user/group analogy: changing membership in the resource group changes access for every key assigned to it.
3. Tags — Models, provider connections, and Combos could be tagged with labels such as local, embedding, production-approved, free, or experimental. An API key could then be granted access to one or more tags. This would be the most flexible and dynamic option, although it would need safeguards because adding a tag to a model could immediately grant access to every key using that tag.
   A fourth possibility might be to call them Access Policies, Entitlements, or Model Access Sets. That may describe the concept more precisely than Profiles or Groups. The policy would contain the allowed resources, and API keys would reference the policy rather than receiving copied permission lists.

In RBAC terms, the API keys are the users or service accounts, the models/Combos/connections are the resources, and the proposed Profile or Policy is the reusable role sitting between them.

I currently work around this by asking an agent to update many keys for me. That works, but it still requires giving the agent fairly broad management access to OmniRoute. I think a native reusable policy would be safer, easier to audit, and useful for anyone operating more than a handful of application-specific keys.

If Diego supports the general concept, I’d mainly want his and everyone else’s input on which structure fits OmniRoute best. I’m less attached to the name or implementation than I am to the core behavior: define access once, attach it to many keys, and have future changes propagate automatically.

I hope that explains the distinction better.

[diegosouzapw]

@seanford Most of this exists already. It just has no dashboard page.

API key groups (src/lib/db/apiKeyGroups.ts, added in migration 065) let you create a group, give it allow/deny model rules (* wildcards work, provider is optional) and put as many keys in it as you want. Deny wins. A key that belongs to a group only gets the models some active group allows. Right now you manage them over REST, under /api/keys/groups. The dashboard page is the missing part, and #16417 lists it for v4. One gap: rules match model patterns, so provider connections aren't covered.

If you want to put time into this, I'd build on groups, starting with a UI for them, rather than add a separate profile or tag model.