Reference governance.yaml for the proxy — eat-our-own-cooking example? #30518
VamsiSudhakaran1
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi team!
litellm is the tool tens of thousands of teams use to put cost controls, rate limiting, and guardrails in front of their LLM calls — so you're closer to this problem than almost anyone.
I ran release-gate (an AI-deployment-readiness scorer) against this repo and it scored 60/100 (HOLD). It correctly detected the safeguards the proxy already supports:
✅ Budget / cost ceiling
✅ Kill switch / fallback
✅ Auth & rate limiting
✅ Eval evidence
Gaps flagged: a formal governance.yaml, a team-owner/on-call field, and a trace/tool policy.
Would you be open to a small PR adding a reference governance.yaml for the proxy deployment — a worked example your enterprise users could copy to document budgets/auth/fallback for their own gateway? Feels on-brand given the product, but I'll only open it if it's useful to you.
Beta Was this translation helpful? Give feedback.
All reactions