Initial OIDC support (Google/GitHub/CircleCI -> Amazon Bedrock & Azure OpenAI) #3507
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Manouchehri
changed the title
4 tasks
| @@ -2089,6 +2090,10 @@ def set_client(self, model: dict): | |||
| raise ValueError( | |||
| f"api_base is required for Azure OpenAI. Set it on your config. Model - {model}" | |||
| ) | |||
| azure_ad_token = litellm_params.get("azure_ad_token") | |||
| if azure_ad_token is not None: | |||
| if azure_ad_token.startswith("oidc/"): | |||
There was a problem hiding this comment.
@Manouchehri can we add a simple unit test in test_router.py to make sure this is always respected?
We've had previous regressions due to untested flows (thinking of cloudflare). Would help to have something for this
| print(f"secret_val: {redact_oidc_signature(secret_val)}") | ||
|
|
||
|
|
||
| @pytest.mark.skipif(os.environ.get('CIRCLE_OIDC_TOKEN_V2') is None, reason="Cannot run without being in a CircleCI Runner") |
There was a problem hiding this comment.
is there a way for us to add unit tests that can run on our ci/cd? @Manouchehri
There was a problem hiding this comment.
The Circle CI ones should already work automatically!
There was a problem hiding this comment.
oh - how? we don't have these tokens
There was a problem hiding this comment.
There was a problem hiding this comment.
They are set by default. That's part of the ODIC magic.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Implement OIDC for Azure OpenAI and Amazon Bedrock requests.
Currently, this supports using OIDC from:
And this supports using the auth token for:
Relevant issues
Resolves #3505 and #3401.
Type
🆕 New Feature
🐛 Bug Fix
💻 Development Environment
🚄 Infrastructure
Changes
This allows using OIDC with Azure OpenAI.
Testing
You must set the following two env vars:
And see #3499 for the AWS IAM info.
Notes
Docs will be in a later PR.
Pre-Submission Checklist (optional but appreciated):
OS Tests (optional but appreciated):