feat(openai): add withAWSBedrock for OpenAI models on Amazon Bedrock#1725
Open
piyush-gambhir wants to merge 2 commits into
Open
feat(openai): add withAWSBedrock for OpenAI models on Amazon Bedrock#1725piyush-gambhir wants to merge 2 commits into
piyush-gambhir wants to merge 2 commits into
Conversation
🦋 Changeset detectedLatest commit: 220a89d The changes in this PR will be included in the next version bump. This PR includes changesets to release 34 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Mirror the Python livekit-agents Bedrock support. Adds `LLM.withAWSBedrock` (Chat Completions, gpt-oss) and `responses.LLM.withAWSBedrock` (Responses API, gpt-5.5 / gpt-5.4), both building an `openai` BedrockOpenAI client. gpt-oss is routed to the mantle `/v1` path; gpt-5.x to `/openai/v1`. - add BedrockChatModels / BedrockResponsesModels and resolveBedrockBaseURL - bump openai to ^6.41.0 (first release exporting BedrockOpenAI); the lockfile grows as pnpm re-resolves the dependency graph for the bump - declare AWS_* env vars in turbo.json - add a changeset and hermetic tests
b11acb2 to
76b18b2
Compare
…rockBaseURL Addresses review feedback. The override was already applied at runtime — `BedrockOpenAI` reads `AWS_BEDROCK_BASE_URL` from its own `baseURL` default when `undefined` is passed — but the function returning `undefined` there read like a bug. Return the env value (and an explicit `baseURL`) directly for every model so the resolution is self-evident, and add a regression test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds Amazon Bedrock support to the OpenAI plugin, porting the Python
livekit-agentswork (livekit/agents#5978). Bedrock's OpenAI-compatiblebedrock-mantleendpoint serves two model families on different paths, so this adds awithAWSBedrock(...)factory to both the Chat Completions and Responses LLMs:gpt-oss-120b,gpt-oss-20b/v1LLM.withAWSBedrock(...)gpt-5.5,gpt-5.4/openai/v1responses.LLM.withAWSBedrock(...)Both build an
openaiBedrockOpenAIclient. openai-node only derives the/openai/v1path (gpt-5.x); thegpt-ossmodels live on/v1, so the path is resolved from the model id. The Responses variant disables the WebSocket transport automatically, since Bedrock only exposes the HTTP Responses path.apiKey/awsRegion/baseURLare inferred fromAWS_BEARER_TOKEN_BEDROCK/AWS_REGION(orAWS_DEFAULT_REGION) /AWS_BEDROCK_BASE_URL.apiKeyandbedrockTokenProvider(refreshable credentials) are mutually exclusive.Changes Made
LLM.withAWSBedrock(Chat Completions) andresponses.LLM.withAWSBedrock(Responses API)BedrockChatModels/BedrockResponsesModelstypes and theresolveBedrockBaseURLhelper (routesgpt-oss→/v1)AWS_*env vars inturbo.json(required byturbo/no-undeclared-env-vars)minor) and hermetic tests (plugins/openai/src/bedrock.test.ts)openaito^6.41.0— the first release exportingBedrockOpenAI. This is whypnpm-lock.yamlhas a large diff: the lock is otherwise stable under pnpm 9.7, but bumpingopenaimakes pnpm re-resolve the dependency graph (the monorepo pins multiplevitestversions, so their transitive snapshots get re-expanded).pnpm install --frozen-lockfilepasses against the updated lock.Pre-Review Checklist
Testing
plugins/openai/src/bedrock.test.ts(construction,/v1vs/openai/v1routing, credential mutual-exclusion)pnpm exec vitest run plugins/openai/src/bedrock.test.ts→ 7 passed;pnpm build(incl.tscdeclarations) clean; eslint/prettier clean;pnpm install --frozen-lockfilevalidrestaurant_agent.ts/realtime_agent.ts— N/A (no voice-pipeline change)Additional Notes
The Python counterpart (livekit/agents#5978) was verified end-to-end against live Bedrock (gpt-5.5/5.4 via Responses, gpt-oss-120b via Chat Completions), confirming the model IDs, the
/v1routing, and bearer-token auth.