Skip to content

feat(openai): add withAWSBedrock for OpenAI models on Amazon Bedrock#1725

Open
piyush-gambhir wants to merge 2 commits into
livekit:mainfrom
piyush-gambhir:feat/openai-aws-bedrock
Open

feat(openai): add withAWSBedrock for OpenAI models on Amazon Bedrock#1725
piyush-gambhir wants to merge 2 commits into
livekit:mainfrom
piyush-gambhir:feat/openai-aws-bedrock

Conversation

@piyush-gambhir
Copy link
Copy Markdown
Contributor

@piyush-gambhir piyush-gambhir commented Jun 6, 2026

Description

Adds Amazon Bedrock support to the OpenAI plugin, porting the Python livekit-agents work (livekit/agents#5978). Bedrock's OpenAI-compatible bedrock-mantle endpoint serves two model families on different paths, so this adds a withAWSBedrock(...) factory to both the Chat Completions and Responses LLMs:

Models API on Bedrock Mantle path Factory
gpt-oss-120b, gpt-oss-20b Chat Completions (+ Responses) /v1 LLM.withAWSBedrock(...)
gpt-5.5, gpt-5.4 Responses only /openai/v1 responses.LLM.withAWSBedrock(...)

Both build an openai BedrockOpenAI client. openai-node only derives the /openai/v1 path (gpt-5.x); the gpt-oss models live on /v1, so the path is resolved from the model id. The Responses variant disables the WebSocket transport automatically, since Bedrock only exposes the HTTP Responses path.

import { LLM, responses } from '@livekit/agents-plugin-openai';

const a = LLM.withAWSBedrock({ model: 'openai.gpt-oss-120b', awsRegion: 'us-east-2' });          // chat
const b = responses.LLM.withAWSBedrock({ model: 'openai.gpt-5.5', awsRegion: 'us-east-2' });     // responses

apiKey / awsRegion / baseURL are inferred from AWS_BEARER_TOKEN_BEDROCK / AWS_REGION (or AWS_DEFAULT_REGION) / AWS_BEDROCK_BASE_URL. apiKey and bedrockTokenProvider (refreshable credentials) are mutually exclusive.

Changes Made

  • LLM.withAWSBedrock (Chat Completions) and responses.LLM.withAWSBedrock (Responses API)
  • BedrockChatModels / BedrockResponsesModels types and the resolveBedrockBaseURL helper (routes gpt-oss/v1)
  • Declared AWS_* env vars in turbo.json (required by turbo/no-undeclared-env-vars)
  • Added a changeset (minor) and hermetic tests (plugins/openai/src/bedrock.test.ts)
  • Bumped openai to ^6.41.0 — the first release exporting BedrockOpenAI. This is why pnpm-lock.yaml has a large diff: the lock is otherwise stable under pnpm 9.7, but bumping openai makes pnpm re-resolve the dependency graph (the monorepo pins multiple vitest versions, so their transitive snapshots get re-expanded). pnpm install --frozen-lockfile passes against the updated lock.

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title (the lockfile growth is explained above)
  • Video demo: N/A — this adds LLM constructors, no pipeline/UI behavior change

Testing

  • Automated tests added/updated — plugins/openai/src/bedrock.test.ts (construction, /v1 vs /openai/v1 routing, credential mutual-exclusion)
  • All tests pass — pnpm exec vitest run plugins/openai/src/bedrock.test.ts → 7 passed; pnpm build (incl. tsc declarations) clean; eslint/prettier clean; pnpm install --frozen-lockfile valid
  • restaurant_agent.ts / realtime_agent.ts — N/A (no voice-pipeline change)

Additional Notes

The Python counterpart (livekit/agents#5978) was verified end-to-end against live Bedrock (gpt-5.5/5.4 via Responses, gpt-oss-120b via Chat Completions), confirming the model IDs, the /v1 routing, and bearer-token auth.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Jun 6, 2026

🦋 Changeset detected

Latest commit: 220a89d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 34 packages
Name Type
@livekit/agents-plugin-openai Major
@livekit/agents-plugin-anam Major
@livekit/agents-plugin-cartesia Major
@livekit/agents-plugin-cerebras Major
@livekit/agents-plugin-elevenlabs Major
@livekit/agents-plugin-fishaudio Major
@livekit/agents-plugin-google Major
@livekit/agents-plugin-hume Major
@livekit/agents-plugin-inworld Major
@livekit/agents-plugin-neuphonic Major
@livekit/agents-plugin-perplexity Major
@livekit/agents-plugin-rime Major
@livekit/agents-plugin-sarvam Major
@livekit/agents-plugin-xai Major
@livekit/agents Major
@livekit/agents-plugin-assemblyai Major
@livekit/agents-plugin-baseten Major
@livekit/agents-plugin-bey Major
@livekit/agents-plugin-deepgram Major
@livekit/agents-plugin-hedra Major
@livekit/agents-plugin-lemonslice Major
@livekit/agents-plugin-liveavatar Major
@livekit/agents-plugin-livekit Major
@livekit/agents-plugin-minimax Major
@livekit/agents-plugin-mistral Major
@livekit/agents-plugin-mistralai Major
@livekit/agents-plugin-phonic Major
@livekit/agents-plugin-resemble Major
@livekit/agents-plugin-runway Major
@livekit/agents-plugin-silero Major
@livekit/agents-plugin-soniox Major
@livekit/agents-plugin-tavus Major
@livekit/agents-plugin-trugen Major
@livekit/agents-plugins-test Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

Mirror the Python livekit-agents Bedrock support. Adds `LLM.withAWSBedrock`
(Chat Completions, gpt-oss) and `responses.LLM.withAWSBedrock` (Responses API,
gpt-5.5 / gpt-5.4), both building an `openai` BedrockOpenAI client. gpt-oss is
routed to the mantle `/v1` path; gpt-5.x to `/openai/v1`.

- add BedrockChatModels / BedrockResponsesModels and resolveBedrockBaseURL
- bump openai to ^6.41.0 (first release exporting BedrockOpenAI); the lockfile
  grows as pnpm re-resolves the dependency graph for the bump
- declare AWS_* env vars in turbo.json
- add a changeset and hermetic tests
@piyush-gambhir piyush-gambhir force-pushed the feat/openai-aws-bedrock branch from b11acb2 to 76b18b2 Compare June 6, 2026 05:33
…rockBaseURL

Addresses review feedback. The override was already applied at runtime —
`BedrockOpenAI` reads `AWS_BEDROCK_BASE_URL` from its own `baseURL` default when
`undefined` is passed — but the function returning `undefined` there read like a
bug. Return the env value (and an explicit `baseURL`) directly for every model so
the resolution is self-evident, and add a regression test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant