packages ai providers

Providers

Active contributors: Mario Zechner, kt, Armin Ronacher

Purpose

Every provider implementation lives in packages/ai/src/providers/. Each one exports a stream function and a streamSimple function that return an AssistantMessageEventStream for one Api identifier, plus a provider-specific options interface. The registry (packages/ai/src/api-registry.ts) and the lazy loader (packages/ai/src/providers/register-builtins.ts) treat them uniformly, so the rest of the repo only ever deals with Model, Context, and the event stream.

All providers follow the stream contract from packages/ai/src/types.ts: push start, emit text_delta / thinking_delta / toolcall_delta events while a block streams, then terminate with done or error. Failures are classified and recorded through packages/ai/src/utils/stream-failure.ts instead of being thrown.

Provider table

Provider file	API identifier	Auth	Notes
packages/ai/src/providers/anthropic.ts	anthropic-messages	API key (ANTHROPIC_API_KEY) or OAuth token (ANTHROPIC_OAUTH_TOKEN)	Uses the Anthropic SDK. Prompt caching via cache_control with short (5m) or long (1h) retention, thinking levels with token budgets, interleaved tool-result streaming, and Claude Code tool-name casing mimicry for compatibility
packages/ai/src/providers/openai-responses.ts	openai-responses	OPENAI_API_KEY	OpenAI Responses API. Supports 24h prompt-cache retention and a session_id cache-affinity header; also serves OpenRouter-style routing options through compat
packages/ai/src/providers/openai-completions.ts	openai-completions	Provider-specific keys (e.g. DEEPSEEK_API_KEY, GROQ_API_KEY, XAI_API_KEY, OPENROUTER_API_KEY, ZAI_API_KEY, HF_TOKEN, PRIME_API_KEY)	Chat Completions API reused by many OpenAI-compatible providers. Behavior is auto-detected from baseUrl and overridable per model through OpenAICompletionsCompat (thinking format, max-token field, tool-result name requirements, cache format)
packages/ai/src/providers/openai-codex-responses.ts	openai-codex-responses	ChatGPT subscription via OAuth (see packages/ai/src/utils/oauth/openai-codex.ts)	Codex backend at chatgpt.com/backend-api. Session-based conversation continuity, WebSocket transport option, JWT re-auth, retry with exponential backoff, and session resource cleanup
packages/ai/src/providers/azure-openai-responses.ts	azure-openai-responses	AZURE_OPENAI_API_KEY plus resource, API version, and deployment name	Azure OpenAI Responses. Deployment names resolved per model via AZURE_OPENAI_DEPLOYMENT_NAME_MAP
packages/ai/src/providers/google.ts	google-generative-ai	GEMINI_API_KEY	Gemini via @google/genai GenerateContent streaming. Thinking budgets, thought signatures for multi-turn continuity, tool choice control
packages/ai/src/providers/google-vertex.ts	google-vertex	API key (GOOGLE_CLOUD_API_KEY) or Application Default Credentials plus project and location	Vertex AI Gemini via @google/genai. Same options as the Google provider plus project and location
packages/ai/src/providers/amazon-bedrock.ts	bedrock-converse-stream	AWS credentials (profile, IAM keys, ECS task roles, IRSA web identity) or bearer token (AWS_BEARER_TOKEN_BEDROCK)	AWS SDK ConverseStream. Thinking display modes (summarized / omitted), interleaved thinking for Claude 4.x, cache points, request metadata tags. Node-only; loadable in Bun via setBedrockProviderModule (packages/ai/src/bedrock-provider.ts)
packages/ai/src/providers/mistral.ts	mistral-conversations	MISTRAL_API_KEY	Mistral SDK chat.stream. Reasoning prompt mode and tool choice
packages/ai/src/providers/faux.ts	faux	none	In-memory fake provider for tests. registerFauxProvider installs scripted responses, tool calls, and failure cases without any network

Shared helpers in the same directory:

File	Role
packages/ai/src/providers/register-builtins.ts	Lazy registration of all built-in APIs. Must never statically import provider implementation modules
packages/ai/src/providers/openai-responses-shared.ts	Message and tool conversion plus stream processing shared by the Responses-style providers (OpenAI, Codex, Azure)
packages/ai/src/providers/google-shared.ts	Google message conversion, thinking-part mapping, budgets, and tool choice shared by the Google and Vertex providers
packages/ai/src/providers/transform-messages.ts	Cross-provider message normalization: image downgrade for non-vision models, thinking-block handling, tool-call ID normalization, synthetic tool results for orphaned calls
packages/ai/src/providers/simple-options.ts	buildBaseOptions, thinking budget defaults, and max-token adjustment for thinking providers
packages/ai/src/providers/cloudflare.ts	Cloudflare base-URL constants and {VAR} placeholder resolution used by the Anthropic and OpenAI providers
packages/ai/src/providers/github-copilot-headers.ts	Copilot-specific headers: X-Initiator, Openai-Intent, Copilot-Vision-Request

streamSimple support

Every built-in provider exports both stream (full options) and streamSimple (only SimpleStreamOptions plus a reasoning level). streamSimple is what the agent loop calls by default (packages/agent/src/agent-loop.ts falls back to streamSimple), and it maps the reasoning level onto provider-specific mechanisms: token budgets for token-based providers via adjustMaxTokensForThinking in packages/ai/src/providers/simple-options.ts, thinking_effort for OpenAI-style APIs, or Google thinking config. The SimpleStreamOptions.reasoning type is a ThinkingLevel (minimal through max); providers without a distinct xhigh/max tier clamp them to high.

Lazy registration pattern

packages/ai/src/providers/register-builtins.ts is the only file in the package that knows about all providers, and it never imports them statically. Each provider has a loader that memoizes a module promise:

anthropicProviderModulePromise ||= import("./anthropic.js").then((module) => ({ ... }));

createLazyStream and createLazySimpleStream wrap a loader in a StreamFunction: they create an outer AssistantMessageEventStream, await the module, forward the inner stream's events into it, and on a module-load error push an error event carrying a message with stopReason "error" and the load error text. This keeps loading the package entry cheap and keeps node-only providers (notably Bedrock) out of browser bundles. setBedrockProviderModule lets a host replace the Bedrock implementation, which is how Bun loads it via packages/coding-agent/src/bun/register-bedrock.ts.

Adding a provider

1. Implement stream and streamSimple plus an options interface in a new file under packages/ai/src/providers/.
2. Add a lazy loader and a registerApiProvider call in packages/ai/src/providers/register-builtins.ts.
3. Add the env var to packages/ai/src/env-api-keys.ts.
4. Add a subpath export in packages/ai/package.json and, if the options type should be public, a re-export in packages/ai/src/index.ts.
5. Add models through packages/ai/scripts/generate-models.ts (see Models) and a representative model to packages/ai/test/stream.test.ts.

Related pages

• AI package overview: streaming model, registry, and integration points
• Models: the model catalog each provider consumes
• Provider login and model selection: user-facing login and model pickers
• OAuth login flows: subscription OAuth flows
• Glossary: term definitions