Repository navigation
Per-group model access and limits on user-added models #945
gaelpotin-vja
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hi,
We run Cloudflare OS as an internal AI workspace, with Cloudflare Access
in front and AI Gateway with our own provider keys (BYOK). We would like
most users on a cost-efficient default model (cloudflare/auto through the
Auto Router), while letting a small group, such as the IT team, pick
more capable models manually.
What we found in the current release:
Model visibility is deployment-wide. Each gateway model has a single
global mode (enabled / hidden / disabled, stored in
AdminConfig.modelModes). There is no way to make a model visible to
some users or groups only.
"Users may add their own models" cannot be scoped. When enabled, any
user can add a model under any name for any active provider, and that
model runs through the deployment's gateway with the deployment's
keys. Hiding or disabling an expensive model in the catalog does not
prevent a user from adding it themselves. An admin can only turn the
option on or off for everyone.
Our workaround is an AI Gateway dynamic route that checks
metadata.user and sends non-authorized users back to cloudflare/auto.
It works, but the member list lives in the route, outside Cloudflare OS,
and has to be maintained by hand.
What we would like, in order of preference:
Per-model access rules: allow an admin to restrict a catalog model to
a list of users or to groups coming from the identity provider
(for example Cloudflare Access group claims).
Scoping for user-added models: an allowlist of model IDs that users
may add, and the ability to reserve the option to some users or
groups.
Optionally, a per-user or per-group spend limit, building on the
metadata.user already sent to AI Gateway.
Questions:
(AdminConfig fields, admin UI, enforcement point in the backend)?
Happy to test a preview on our instance.
Thanks.
All reactions