v1.1.0

Envoy AI Gateway v1.1.0 is the first minor release on the stable 1.x API. It adds token counting across providers, per-request upstream credentials, stream idle timeout with failover, MCP hostname routing and CEL backend selection, optional OpenTelemetry GenAI tracing, and HTTP CONNECT egress. Upgrading from v1.0 needs no CRD migrations; the only operator-facing change is Helm's restricted controller security context.

✨ New Features

Token Counting APIs

• vLLM-compatible /tokenize across providers — Count tokens for a prompt without generating a completion. Chat messages and completion prompts use the vLLM tokenize request shape against vLLM, Vertex AI Gemini, GCP Anthropic, AWS Bedrock Converse, and AWS Anthropic. Native Anthropic clients can use /anthropic/v1/messages/count_tokens; Responses API clients can use /v1/responses/input_tokens (OpenAI and Azure OpenAI).
• Anthropic-format /anthropic/v1/models — Claude clients that call Anthropic's model list endpoint now get the gateway's configured catalog in Anthropic's response shape, scoped by hostname the same way /v1/models is.

Per-Request Credentials & Egress

• Per-request upstream credentials via credentialOverride — A trusted filter can supply the backend credential per request instead of sharing one static key in BackendSecurityPolicy. Source from Envoy dynamic metadata (preferred) or a request header the gateway strips before the backend. API keys, Anthropic, Azure, GCP, and AWS SigV4 are supported.
• HTTP CONNECT forward proxy — GatewayConfig.spec.forwardProxy tunnels upstream LLM connections through an HTTP CONNECT proxy (host:port). Bedrock requests through PrivateLink now sign over the actual upstream host, so VPC endpoints no longer fail with a silent 403.

Streaming Resilience

• Stream idle timeout with failover — AIGatewayRouteRule.streamIdleTimeout bounds how long Envoy waits without upstream bytes on a streaming response. If it fires before the first token, a retry policy can fail over to the next backend; mid-stream it returns 504. Pair with a BackendTrafficPolicy whose retry covers reset.

MCP Gateway

• Hostname routing and backend selection — MCPRoute.spec.hostnames scopes an MCP endpoint to specific hosts. backendSelector restricts which backends a session fans out to via CEL rules evaluated at initialize (default Deny). Initialize now advertises merged backend capabilities, and the controller shares one Envoy Gateway Backend per namespace.

Observability

• OpenTelemetry GenAI conventions and Grafana dashboard — Set AI_GATEWAY_TRACING_SEMCONV=gen_ai on the ext-proc to emit gen_ai.* span attributes. An example dashboard for gen_ai_* Prometheus metrics ships in examples/monitoring/grafana-dashboard.json. Controller and ext-proc logs accept --logFormat=json; MCP resource reads populate mcp_resource_uri in access-log metadata.

Provider Translation & API Compatibility

• Structured output for Claude on Vertex AI — JSON-schema constrained decoding now works for Claude models on Vertex AI that advertise output_config. OpenAI reasoning_effort is forwarded to Bedrock as reasoning_config.
• Responses API tool_search, unknown tools, and Codex inputs — The Responses API accepts the built-in tool_search tool and additional_tools, preserves unknown tool types, and accepts Codex-style agent input items.

Helm & Operations

• Controller hardening and sharded config secrets — The chart can emit a PDB, honour topologySpreadConstraints, and attach extra podLabels. Filter configuration is split across multiple Secrets so large Gateway configs no longer hit Kubernetes' 1 MiB Secret size limit.

🔗 API Updates

• AIGatewayRouteRule.name — Optional unique name copied onto the generated HTTPRoute rule. route-not-found is reserved.
• AIGatewayRouteRule.streamIdleTimeout — Optional Gateway API duration; unset means no per-try idle timeout.
• BackendSecurityPolicy.spec.credentialOverride — Optional per-request credential source. Exactly one of fromRequestHeaders or fromDynamicMetadata. fallbackToConfigured defaults to true.
• GatewayConfig.spec.forwardProxy — Optional HTTP CONNECT proxy. address is required (host:port).
• MCPRoute.spec.hostnames — Optional list of hostnames (max 16).
• MCPRoute.spec.backendSelector — Optional CEL selector evaluated at session initialize. defaultAction defaults to Deny.

⚠️ Breaking Changes

• Helm controller security context defaults — The Helm chart now sets a restricted podSecurityContext and container securityContext on the controller (non-root UID/GID 65532, all capabilities dropped, no privilege escalation, RuntimeDefault seccomp). Previously both were empty. If you run a custom controller image that cannot use UID 65532, override controller.podSecurityContext and controller.securityContext in your values.

📖 Upgrade Guidance

Upgrading from v1.0 requires no CRD migrations. New fields are optional and backward compatible. Review the Helm security-context change if you override the controller image.

If the official controller image works for you, no action is needed. To restore empty security contexts:

controller:
  podSecurityContext: {}
  securityContext: {}

JSON logs: set controller.logFormat and extProc.logFormat to json. Enable a PDB with controller.podDisruptionBudget.enabled: true when you run more than one replica.

📦 Dependency Versions

Dependency	Version
Go	1.26.4
Envoy Gateway	v1.8.1
Envoy Proxy	v1.38.1
Gateway API	v1.5.1
Gateway API Inference Extension	v1.0.2
MCP Go SDK	v1.7.0

🙏 Acknowledgements

Thank you to everyone who made v1.1 possible.

v1.0.0

Envoy AI Gateway v1.0.0 — General Availability

Envoy AI Gateway v1.0.0 marks General Availability. With this release the core control-plane API — AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute, all served at v1beta1 — is declared stable: within the 1.x series we will not make breaking changes to it unless required by a critical security fix, and any such change will ship with a documented migration path. Upgrading from v0.7 requires no changes to your resources. 1.0 brings together everything built since the first release in February 2025: a single OpenAI-compatible API across 16 providers with cross-provider translation, a full Model Context Protocol gateway, multimodal and audio endpoints, enterprise-grade observability, and multi-tenant, quota-aware routing — all as an additive layer on CNCF Envoy Gateway.

🎉 What 1.0 Means

1.0 is a commitment, not just another feature release. From the first release we said the major version would arrive once we had a first stable control-plane API. That moment is here. General Availability means:

• A stable API. Your v1beta1 resources will not break under you within the 1.x series.
• Predictable upgrades. Upgrading the controller will not break a valid, migrated configuration; any change requiring action ships with a documented path.
• A complete platform. Everything assembled since v0.1 is now production-ready on the proven Envoy Gateway foundation.

Our API stability commitment

For stable releases, we will never break the APIs unless there is a critical security issue, and we will always provide a migration path in the release notes if we ever must. Following Semantic Versioning, the v1beta1 control-plane API remains backward compatible for the entire 1.x series — breaking changes would only ever land in a future 2.0. See the full support policy.

✨ The 1.0 Feature Surface

These are the capabilities the stable 1.0 control plane brings together.

A Stable, Versioned Control-Plane API

• Core CRDs now covered by the stability guarantee — The control-plane API you build on — AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, and MCPRoute, all served at v1beta1 — is now a stable contract. Within the 1.x series these APIs will not change in a breaking way unless required by a critical security fix, and any such change will ship with a documented migration path.

Universal LLM Access

• One OpenAI-compatible API across 16 providers — Reach OpenAI, Azure OpenAI, Google Gemini, Google Vertex AI, AWS Bedrock, Anthropic, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, and the Tetrate Agent Router Service through a single endpoint. Switch or mix providers without changing client code.
• Cross-provider request/response translation — Translate between provider protocols transparently — Anthropic /v1/messages to OpenAI /v1/chat/completions, and Anthropic Messages to AWS Bedrock Converse and InvokeModel — including streaming, tool use, reasoning/thinking blocks, and images.
• Model virtualization with modelNameOverride — Expose stable, application-facing model names while the gateway maps them to provider-specific models, enabling A/B testing, gradual migrations, and multi-provider strategies without touching client code.

Full Endpoint Coverage

• Chat, completions, embeddings, and images — /v1/chat/completions, /v1/completions, /v1/embeddings, and /v1/images/generations across compatible providers.
• Audio: transcription, translation, and speech — /v1/audio/transcriptions, /v1/audio/translations, and /v1/audio/speech bring speech-to-text and text-to-speech workloads through the gateway.
• OpenAI Responses API and multimodal inputs — /v1/responses is supported, including on Azure OpenAI backends, and chat requests accept image, audio_url, and video_url content parts for compatible backends.

MCP Gateway

• Aggregate and route Model Context Protocol servers — Multiplex multiple MCP servers behind one endpoint with MCPRoute, including tool routing and include/exclude filtering.
• Fine-grained, CEL-based authorization — Enforce per-tool authorization. tools/list applies the same rules as tools/call, so callers only discover the tools they are allowed to invoke.
• Per-backend header forwarding with JWT claim projection — Forward selected request headers and project JWT claims to individual MCP backends.

Traffic Management & Multi-Tenancy

• Hostname-based multi-tenant routing — Serve different model sets per hostname from a single Gateway with AIGatewayRoute.spec.hostnames; the /v1/models endpoint scopes its response to the matching host.
• Token- and quota-aware rate limiting — Rate limit on model tokens and per QuotaPolicy, with backend rate limit filter injection to enforce quota-based throttling.
• Provider fallback and InferencePool support — Automatic failover across providers, plus intelligent endpoint selection for self-hosted models via the Gateway API Inference Extension.

Provider Authentication & Compliance

• BackendSecurityPolicy for upstream authentication — Centralize provider credentials with API key, AWS, Azure, and GCP cloud-native identity, including GKE Workload Identity via Application Default Credentials.
• Request/response body redaction — Redact sensitive request and response bodies to meet compliance requirements.

Enterprise Observability

• OpenTelemetry tracing with OpenInference — Full request-lifecycle tracing, compatible with AI evaluation tools like Arize Phoenix.
• GenAI token metrics and reasoning-token accounting — Prometheus metrics for token usage, time-to-first-token, and inter-token latency, with separate accounting for reasoning tokens.

🔗 API Updates

• The v1beta1 API is now stable — v1.0 does not change the API surface. Instead it elevates the existing v1beta1 CRDs to a stable contract under our support policy: no new apiVersion is introduced and no resource migration is required. New fields added during the 1.x series will remain backward compatible.

⚠️ Breaking Changes

None. v1.0 introduces no breaking changes. The v1beta1 API is unchanged — 1.0 declares it stable rather than altering it — so there is no apiVersion bump and no resource migration. If you are running v0.7, your existing resources work as-is.

🛡️ Support & Compatibility Policy

With 1.0, the project's support policy applies in full:

• API compatibility. The v1beta1 CRDs are stable for the 1.x series. New fields are added in a backward-compatible way; breaking changes are reserved for a future major version and would ship with a migration path.
• Controller upgrades. Upgrading the controller will not break a valid configuration. Upgrade at most two minor versions at a time, following any documented migration steps.
• Envoy Gateway compatibility. Each release is built on the latest stable Envoy Gateway (and therefore Envoy Proxy); keep Envoy Gateway up to date before upgrading Envoy AI Gateway.
• End of life. A release is supported until two releases after it, consistent with prior versions.

📖 Upgrade Guidance

Upgrading from v0.7 is a drop-in change — there are no API or resource changes:

1. Update the Helm chart / controller image to the v1.0.0 release.
2. Roll out as usual. Your existing v1beta1 resources require no edits.

If you are on an older release, upgrade one or two minor versions at a time and follow the migration steps in each series' release notes (notably the v0.6 promotion of the core CRDs to v1beta1) before moving to 1.0.

📦 Dependency Versions

Dependency	Version
Go	1.26.4
Envoy Gateway	v1.8.1
Envoy Proxy	v1.38.1
Gateway API	v1.5.1
Gateway API Inference Extension	v1.0.2
MCP Go SDK	v1.6.1

🙏 Acknowledgements

1.0 belongs to everyone who got us here. Our deepest thanks to:

• The maintainers across Tetrate, Bloomberg, Tencent, and Nutanix, and the many independent contributors who shaped the project through code, reviews, and weekly community meetings.
• The early adopters — including Bloomberg, LY Corporation, Alan by Comma Soft, and NRP — who ran Envoy AI Gateway in production and fed back what mattered.
• The broader Gateway API, Envoy, and CNCF communities whose standards this project is built on.

🔮 What's Next

A stable API is a starting line, not a finish line. On the roadmap:

• A dedicated MCPBackend CRD, decoupling MCP backend configuration from MCPRoute.
• Deeper MCP authorization across tools, resources, and prompts.
• Fuller quota-aware routing that automatically steers around rate-limited upstreams.
• More provider translation paths and expanded multimodal support.

v1.0.0-rc1

Release candidate

v0.7.0

Envoy AI Gateway v0.7.0

Envoy AI Gateway v0.7.0 adds hostname-based routing to AIGatewayRoute, enabling multi-tenant deployments where different hostnames expose different model sets through a single Gateway. A new Anthropic Messages → AWS Bedrock Converse translator lets Anthropic-native clients reach Bedrock without switching protocols. OpenAI audio transcription and translation endpoints arrive alongside Azure OpenAI Responses API support. Quota-aware rate limiting takes its first step with backend rate limit filter injection for QuotaPolicy. Claude Opus 4.7 gains full reasoning support including the display parameter and xhigh effort tier. Anthropic-to-OpenAI translation now handles reasoning blocks and images end-to-end. MCP tools/list responses respect authorization rules, and multimodal support grows with audio_url and video_url content types. Several SSE streaming and provider translation bugs are fixed.

✨ New Features

Multi-Tenant Hostname Routing

• Hostname-based model scoping on AIGatewayRoute — Serve different model sets from a single Gateway by assigning hostnames to each AIGatewayRoute. The /v1/models endpoint automatically returns only the models declared by routes matching the request's Host header, so tenants on teamA.ai.example.com and teamB.ai.example.com each see their own catalog without separate Gateways. Wildcard hostnames (*.ai.example.com) are supported following the Gateway API hostname matching rules.

Provider Translation

• Anthropic /v1/messages → AWS Bedrock Converse API — Send requests in Anthropic Messages format and have them translated to Bedrock's Converse and ConverseStream APIs automatically. Supports text, images, tool use, thinking blocks, and streaming — so Anthropic-native clients can reach any Bedrock model without changing their integration. Complements the existing OpenAI → Bedrock Converse and Anthropic → Bedrock InvokeModel paths.
• Reasoning and image support for Anthropic-to-OpenAI translation — The Anthropic /v1/messages → OpenAI /v1/chat/completions path now handles thinking/reasoning content and image blocks end-to-end. Thinking config (enabled/disabled/adaptive) passes through, thinking and redacted_thinking blocks are preserved in multi-turn conversations, and image blocks (base64 and URL) convert to OpenAI image_url format. Previously these were silently dropped.
• Claude Opus 4.7 and Mythos Preview reasoning — Full support for Claude Opus 4.7's reasoning features: the display parameter (summarized/omitted) controls thinking content visibility, and xhigh joins the reasoning effort tiers for long-horizon agentic and coding tasks. Both claude-opus-4-7 and claude-mythos-preview models are recognized for effort-based thinking control.
• Custom request paths for Anthropic backends via prefix — The prefix field on VersionedAPISchema now works for Anthropic-schema backends, producing endpoints like /{prefix}/messages instead of the default /v1/messages. Useful for routing to Anthropic-compatible providers that use a non-standard path.
• Anthropic anthropic-beta header forwarded to AWSAnthropic — The anthropic-beta request header is now mapped into the anthropic_beta body field when routing to AWSAnthropic backends, so beta features like extended thinking and token counting work through the gateway without manual body rewriting.

OpenAI API Compatibility

• Audio transcription and translation endpoints — Full data-plane support for OpenAI's /v1/audio/transcriptions (Whisper transcription) and /v1/audio/translations (Whisper translation) endpoints. These accept multipart/form-data requests containing audio files, enabling speech-to-text workloads to flow through the gateway with the same auth, rate limiting, and observability as other traffic.
• Azure OpenAI Responses API — The OpenAI-compatible /v1/responses endpoint now works with Azure OpenAI backends, routing requests to Azure's /openai/responses?api-version=... path while preserving existing request and response handling. Azure users get Responses API support without changing client code.
• audio_url and video_url content types — OpenAI chat completion requests can now include audio_url and video_url content parts, enabling multimodal audio and video inputs for compatible backends like vLLM with phi-4-mm and Qwen 3.5 models.

Quota-Aware Routing

• Backend quota rate limit filter injection — First step toward quota-aware routing: the controller now injects a backend rate limit filter when a QuotaPolicy is attached to an AIServiceBackend. The QuotaPolicy controller reconciles the policy, builds rate limit descriptor trees, and configures the rate limit service. This enables per-backend request throttling based on upstream provider quotas.

MCP Gateway

• Authorization-filtered tools/list responses — MCP tools/list now applies the same authorization rules used by tools/call, omitting tools the caller isn't authorized to invoke. Prevents unauthorized callers from discovering tool names and avoids wasted LLM turns on tools that would fail at call time.

Observability

• Smarter log redaction preserves developer-authored metadata — Debug log redaction (--enableRedaction) no longer masks developer-authored schema metadata that was previously over-redacted: tool definition description and parameters, tool call function.name, response_format.json_schema, and guided_json are now visible in debug logs. User-provided content and AI-generated text remain redacted, making debug logs significantly more useful without compromising privacy.

🔗 API Updates

• AIGatewayRoute.spec.hostnames — New optional field accepting a list of hostnames for hostname-based request filtering. When specified, the generated HTTPRoute includes these hostnames, and the /v1/models endpoint scopes its response to models from matching routes. Follows Gateway API hostname semantics including wildcard support.
• AIGatewayRoute.spec.rules capped at 15 — Maximum rules per AIGatewayRoute reduced from 128 to 15 to match the Gateway API HTTPRoute limit (one slot is reserved for a controller-injected catch-all rule). To configure more rules on the same Gateway, split them across multiple AIGatewayRoute resources.
• VersionedAPISchema.prefix supported for Anthropic — The prefix field now applies to Anthropic-schema backends in addition to OpenAI. The version field is ignored for Anthropic; use prefix for custom paths. Note: prefix is ignored for AWSAnthropic and GCPAnthropic as these override paths internally.
• QuotaPolicy rate limit filter injection (runtime enforcement) — The QuotaPolicy CRD (introduced as API-only in v0.6) now has its first runtime behavior: when attached to an AIServiceBackend, a backend rate limit filter is injected to enforce quota-based throttling. Full quota-aware routing across multiple backends is planned for future releases.

🐛 Bug Fixes

• SSE parser handles fields without space after colon — The SSE event parser now correctly handles fields formatted as data:{json} (no space after the colon), in addition to the standard data: {json}. Fixes silent field drops when proxying responses from providers that omit the optional space.
• Responses API streaming SSE buffering — OpenAI Responses API and speech streaming translators now buffer incomplete SSE events across response body chunks instead of treating each chunk as self-contained. Fixes dropped or mangled events when TCP segment boundaries split an SSE event mid-frame.
• Responses API token usage from incomplete and failed streams — Token usage is now captured from response.incomplete and response.failed SSE events, not just response.completed. Streams that hit max_output_tokens or encounter post-generation failures no longer report zero tokens.
• Nil output guard in AWS Bedrock response translator — Bedrock can return HTTP 200 with no output field (e.g. guardrail interventions or UnknownOperationException). Previously this caused a nil-pointer panic in the ext-proc; now it returns a clean error to the caller.
• Comprehensive Gemini finish-reason mapping — Gemini finish reasons like SAFETY, BLOCKLIST, RECITATION, MALFORMED_FUNCTION_CALL, and others now map to their correct OpenAI equivalents instead of all falling through to content_filter. Unknown reasons map to error rather than silently misreporting as a content filter event.
• Empty delta in GCP Vertex AI streaming chunks — Streaming response chunks from GCP Vertex AI that lack candidate content now emit an empty delta object instead of omitting the field, conforming to the OpenAI streaming contract and fixing parse errors in strict clients.
• Typeless assistant output messages in Responses API — Multi-turn Responses API inputs that include assistant messages without an explicit type: "message" field (e.g. from OpenCode) now parse correctly. Previously these were treated as easy-input messages, causing unmarshalling failures on output_text content blocks.

📖 Upgrade Guidance

Using Hostname-Based Routing

To serve different model sets per hostname, add hostnames to your AIGatewayRoute:

apiVersion: aigateway.envoyproxy.io/v1beta1
kind: AIGatewayRoute
metadata:
  name: team-a-route
spec:
  hostnames:
    - "team-a.ai.example.com"
  rules:
    - matches:
        - headers:
            - name: x-ai-eg-model
              value: gpt-4o
      backendRefs:
        - name: openai-backend

Routes without hostnames remain accessible on all hosts. When at least one route uses hostname scoping, the /v1/models endpoint automatically returns only the models for the matching host.

Rules-Per-Route Lim...

v0.6.0

Envoy AI Gateway v0.6.0

Envoy AI Gateway v0.6.0 marks the first production-ready API surface:

• The core CRDs (AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, GatewayConfig, MCPRoute) are now served at v1beta1.
• AWS Bedrock gains a native InvokeModel path for Claude alongside Titan embeddings via the OpenAI /v1/embeddings contract.
• Gemini gets first-class embeddings and Anthropic-style prefix context caching.
• Cross-provider clients can hit Anthropic's /v1/messages endpoint on any OpenAI-compatible backend, and a single reasoning_effort knob now works across Anthropic, OpenAI, and Gemini.
• Operators get GKE Workload Identity via Application Default Credentials, configurable webhook host networking, request/response body redaction for compliance, and the Go 1.26.2 + Envoy 1.37 + Envoy Gateway 1.7 baseline.

Two breaking changes land in v0.6 — AIGatewayRoute.spec.filterConfig is removed (move to GatewayConfig), and the deprecated version-as-prefix behavior on VersionedAPISchema is removed (use prefix). See Upgrade Guidance below.

📖 Full documentation

⚠️ Breaking Changes

• AIGatewayRoute.spec.filterConfig removed. The filterConfig field on AIGatewayRoute has been removed. Move external-processor configuration (resources, env vars, image overrides) to a GatewayConfig resource referenced from the Gateway via the aigateway.envoyproxy.io/gateway-config annotation. v0.5 deprecated the resources subfield with a pointer to GatewayConfig; v0.6 removes the entire filterConfig struct, so anything still set there must move now. See the upgrade guidance below.
• VersionedAPISchema.version no longer acts as an endpoint prefix for OpenAI-schema backends. The legacy behavior deprecated in v0.5 is gone. Use the prefix field instead (e.g. prefix: /v1beta/openai for Gemini's OpenAI-compatible API, prefix: /compatibility/v1 for Cohere). See the upgrade guidance below.

✨ New Features

AWS Bedrock

• Native InvokeModel API for Claude — Send requests to Claude models on Bedrock through Bedrock's native InvokeModel endpoint, complementing the existing Converse API path. Useful when applications already speak the Anthropic Messages format and want a thin translation layer.
• OpenAI → Bedrock Titan embeddings translation — Call Amazon Titan embedding models on Bedrock through the standard OpenAI /v1/embeddings contract. Switch embedding providers without changing client code. Cohere and other Bedrock embedding models are not yet covered and will follow in a later release.

Anthropic and Cross-Provider Translation

• Anthropic /v1/messages endpoint on OpenAI backends — Expose any OpenAI-compatible backend through Anthropic's Messages API. Lets Claude-style clients reach OpenAI, Azure OpenAI, or any other OpenAI-compatible provider behind the gateway without rewriting requests.
• Structured output for Claude models — Pass JSON schema constraints through to Claude so responses conform to your declared shape. Available on Anthropic and AWS Bedrock Claude backends today; GCP Vertex AI Claude is excluded pending upstream provider support.
• Cleaner handling when max_tokens is omitted on Anthropic requests — Requests without an explicit max_tokens no longer crash the translator; they're forwarded so the provider returns a normal validation error. Removes a long-standing footgun when forwarding OpenAI-shaped requests through the Anthropic path.
• Adaptive thinking for claude-opus-4.6 — Translate Claude's new adaptive thinking mode end-to-end. Adaptive lets the model decide thinking depth per request rather than committing to a fixed budget, so callers can opt in without bespoke provider code.
• Unified reasoning_effort across Anthropic, OpenAI, and Gemini — A single OpenAI-style reasoning_effort value (low/medium/high/xhigh) now maps onto Anthropic's thinking budgets and Gemini 3's thinking controls. One client knob, three providers.

Gemini Provider

• Gemini embeddings translation — Use Gemini embedding models through the OpenAI /v1/embeddings contract, completing Gemini coverage alongside chat completions and Responses.
• Gemini context caching with prefix-style API — Activate Gemini's context caching using the same Anthropic-style cache_control prefix surface already supported elsewhere. Cut input token costs on long, repeated system prompts without a Gemini-specific code path.
• Gemini reasoning surfaced as thinking blocks — Non-streaming Gemini reasoning is now exposed as both string content and structured thinking_blocks, matching the shape clients already use for Anthropic responses. Streaming responses still surface reasoning as string content only.

OpenAI API Compatibility

• Responses API — context management and richer streaming — Second wave of Responses API work fills in context management and improved streaming so the /v1/responses path is closer to parity with /v1/chat/completions. If you held off on /v1/responses due to missing features, retest now.
• Compatibility with open-source Responses API implementations — Improved compatibility with non-OpenAI implementations of the Responses API (e.g. open-source inference servers that expose a /v1/responses endpoint), broadening which Responses-aware clients can sit in front of the gateway.
• Text-to-speech endpoint /v1/audio/speech — Route OpenAI text-to-speech requests through the gateway, so audio workloads benefit from the same auth, rate limiting, and observability as chat traffic.

MCP Gateway

• Per-backend header forwarding with rename — MCPRouteBackendRef.forwardHeaders accepts a list of inbound headers to forward to each backend, optionally renaming them on the way out. Each MCP backend can receive its own set of headers (e.g. trace context, tenant identifiers, per-user auth) without a single route-wide rule.
• JWT claim forwarding to MCP backends — Project verified JWT claims into outbound headers via MCPRouteOAuth.claimToHeaders, enabling identity-aware tool execution at backend MCP servers without re-authenticating downstream.
• Exclude / excludeRegex on tool selectors — MCPToolFilter now supports deny patterns (literal exclude and regex excludeRegex) alongside the existing include rules. Useful when a backend exposes more capabilities than a given route should surface.
• Tool name in access logs and response metadata — Tool invocations now carry the tool name in dynamic metadata (key mcp_tool_name), so per-tool debugging, dashboards, and access-log fields are straightforward to wire up.
• Per-backend capability tracking — The gateway tracks which MCP server feature flags (tools, prompts, resources, logging, completions) each backend supports and merges them across a route. Capability negotiation now reflects what's actually reachable, so clients don't get told a feature is available when no reachable backend implements it.

Authentication and Identity

• GKE Workload Identity via Application Default Credentials — GCP backends now authenticate using the standard ADC chain when neither credentialsFile nor workloadIdentityFederationConfig is set in the BackendSecurityPolicy. Workloads running on GKE pick up Workload Identity automatically — no static service account JSON secret needed.

Security and Privacy

• Request and response body redaction — Strip or mask sensitive fields in request and response bodies before they hit logs, traces, or metrics. Lets you keep observability on while meeting privacy and compliance constraints.

Observability

• OTLP access logging auto-configured by aigw — Standalone aigw wires up OTLP access logging out of the box when an OTLP endpoint is configured (via OTEL_EXPORTER_OTLP_ENDPOINT), removing a manual step from local-dev and demo paths.
• Default agent-session-id → session.id header mapping — Spans and logs now correlate by session.id automatically when clients send the agent-session-id header, so agent frameworks like Goose get session correlation with zero config. Override or disable via OTEL_AIGW_REQUEST_HEADER_ATTRIBUTES. Metrics never default to session IDs (high cardinality).
• ReasoningToken cost type — LLMRequestCostType now includes ReasoningToken, so you can budget and bill against thinking tokens separately from input, output, and cache cost types.
• Response model metadata — Responses now carry the resolved upstream model in metadata, which clients and downstream tools can read to confirm exactly which model served a request (useful when routes use model aliasing or fallback).
• OTEL attribute count cap removed for large contexts — Removed the OTEL span attribute count limit so long-context requests no longer have parts of their trace silently dropped.

Operations and Extensibility

• Custom webhook port and host network — The conversion webhook can now bind to a configurable port (controller.mutatingWebhook.port) and run on the host network (controller.hostNetwork), smoothing installs in clusters with restrictive admission webhook networking such as GKE private clusters.
• Lua filter slot after the AI ExtProc stage — Lua filters can now be attached after the AI ExtProc stage in the standard filter chain, so you can do last-mile request shaping (header rewrites, body tweaks) without writing a custom EnvoyExtensionPolicy.
• Route-scoped LLM request costs with global defaults — Set GatewayConfig.spec.globalLLMRequestCosts for fleet-wide defaults and override per-route at AIGatewayRoute.spec.llmRequestCosts. Makes per-tenant or per-backend cost tracking straightforward without per-route boilerplate.

🔗 API Updates

• **Core CRDs promoted to `aigateway.en...

v0.6.0-rc1

Release candidate

v0.5.0

Envoy AI Gateway v0.5.0

Multi-gateway configuration, prompt caching cost savings, fine-grained MCP authorization, OpenAI Responses API, and Google Search grounding for Gemini.

Envoy AI Gateway v0.5.0 makes multi-gateway deployments easier with the new GatewayConfig CRD, cuts costs with prompt caching for AWS Bedrock and GCP Claude, and unlocks fine-grained access control with CEL-based MCP authorization. Developers gain OpenAI Responses API support, Google Search grounding for Gemini, and the ability to mutate request bodies per-route. Under the hood, the switch to sonic JSON processing reduces latency across all requests.

📖 Full documentation

---

✨ New Features

Gateway Configuration

• New GatewayConfig CRD — Gateway-scoped configuration via a new custom resource. Reference it from a Gateway via the aigateway.envoyproxy.io/gateway-config annotation to configure the external processor container (env vars, resource requirements, container settings). Multiple Gateways can share the same GatewayConfig.
• Configurable endpoint prefixes — New prefix field on VersionedAPISchema for backends with non-standard OpenAI-compatible prefixes (e.g., Gemini's /v1beta/openai, Cohere's /compatibility/v1).

OpenAI API Support

• OpenAI Responses API (/v1/responses) — Full support with streaming and non-streaming modes, function calling, MCP tools, reasoning, multi-turn conversations, multimodal capabilities, token usage tracking, and OpenInference tracing.

Provider Caching Enhancements

• Prompt caching for AWS Bedrock Claude — Reuse cached system prompts with Bedrock Anthropic models. Cache point markers are handled automatically with separate tracking for cache creation and cache hit tokens.
• Prompt caching for GCP Vertex AI Claude — Same cost-saving prompt caching for Claude models on GCP Vertex AI for system prompts and few-shot examples.

MCP Gateway Enhancements

• Fine-grained authorization with CEL, JWT claims, and external auth — Write expressive CEL rules using request attributes (HTTP method, headers, JWT claims, tool names, call arguments), enforce access based on JWT claim values, or delegate to external gRPC/HTTP authorization services.
• Real-time tool list synchronization — MCP clients automatically receive notifications/tools/list_changed when MCPRoutes update, refreshing available tools without reconnection.
• Stdio server proxy in standalone mode — Run command-line MCP tools (e.g., npx-based servers) without code changes via the aigw CLI HTTP proxy.
• Improved OAuth metadata discovery — Well-known endpoints now serve at the MCPRoute path prefix for correct authorization discovery across multiple routes.

Inference Extension

• Security policies for inference pools — Apply BackendSecurityPolicy to InferencePool resources for consistent authentication across dynamically-selected inference endpoints.

Gemini Provider Enhancements

• Google Search grounding — Give Gemini models access to real-time web information via the google_search tool type with domain filtering, blocking confidence thresholds, and time range restrictions.
• Consistent thinking configuration across providers — Same thinking configuration works for both Anthropic and Gemini models for provider-agnostic reasoning features.
• Gemini 3 reasoning and image quality controls — thinking_level (reasoning depth) and media_resolution (image quality vs. speed) with graceful degradation on older Gemini versions.
• Visibility into model reasoning — Thought summaries extracted and surfaced from Gemini responses when thinking is enabled.
• Enterprise web search integration — enterprise_search tool type for grounding responses in organization-specific search infrastructure and data sources.

Traffic Management

• Route-level body mutation — Inject or remove JSON fields in request bodies per-backend using bodyMutation with set and remove operations. Route-level settings override backend defaults.
• AWS Bedrock service tier control — Choose between standard, flex, priority, and reserved tiers for latency-sensitive or cost-optimized workloads with automatic fallback handling.

Observability Enhancements

• Per-provider cost attribution — New gen_ai.provider.name metric attribute for filtering dashboards and alerts by provider.
• Full tracing for Anthropic Messages API — OpenInference-compliant tracing for the native /messages endpoint, compatible with Arize Phoenix and OpenTelemetry platforms.
• Cohere Rerank visibility — Full OpenTelemetry support for Cohere's v2 rerank endpoint capturing query, documents, and relevance scores.

Performance and Operations

• Faster request processing with sonic JSON — Migrated to bytedance/sonic for JSON encoding/decoding with measurable latency improvements and lower CPU usage.
• Faster cross-namespace reference validation — Optimized ReferenceGrant indexing reduces controller reconciliation time.
• Improved MCP proxy throughput — HTTP connection reuse across MCP proxy requests eliminates per-request connection overhead. Details →

---

🔗 API Updates

• New GatewayConfig CRD — Gateway-level configuration with extProc.kubernetes for container settings. Reference via aigateway.envoyproxy.io/gateway-config annotation.
• VersionedAPISchema.prefix — New prefix field replaces overloading version for endpoint path customization.
• AIGatewayRouteRuleBackendRef.bodyMutation — New field with set (field/value pairs) and remove (field names) for request body manipulation.
• LLMRequestCostType.CacheCreationInputToken — New cost type for tokens written to cache, separate from CachedInputToken.
• MCPRouteSecurityPolicy authorization fields — New authorization block with defaultAction, rules array (CEL, JWT scopes/claims, tools targeting), and extAuth for external authorization.
• BackendSecurityPolicy.targetRefs expansion — Now accepts InferencePool (inference.networking.x-k8s.io) in addition to AIServiceBackend.

Deprecations

• AIGatewayFilterConfigExternalProcessor.resources — Deprecated. Use GatewayConfig instead. Will be removed in v0.6.
• version field as prefix for OpenAI schema — Deprecated. Use the new prefix field. Legacy behavior will be removed in v0.6.

---

🐛 Bug Fixes

• AWS Bedrock Claude streaming reliability — Streaming responses from Bedrock Claude models now complete correctly without truncation.
• Gemini streaming token counts — Token usage in Gemini streaming responses now matches OpenAI format.
• Multi-chunk Gemini tool calls — Tool calls spanning multiple streaming chunks now have correct indices.
• GCP Claude reasoning content — Reasoning/thinking content correctly passes through for Claude on GCP Vertex AI.
• Zero-weight backend references — Backend references with zero weight no longer cause routing errors.
• Umbrella chart image pull secrets — Helm deployments within umbrella charts correctly inherit global.imagePullSecrets.
• GCP global region backends — Vertex AI backends with global region now work correctly.
• Accurate per-token latency metrics — Fixed integer truncation in time_per_output_token calculation.
• Anthropic token counting — Improved accuracy of input and output token counts for Anthropic models.

---

📖 Upgrade Guidance

Migrating to GatewayConfig

If you're using AIGatewayFilterConfigExternalProcessor.resources, migrate to the new GatewayConfig CRD:

1. Create a GatewayConfig resource:

apiVersion: aigateway.envoyproxy.io/v1alpha1
kind: GatewayConfig
metadata:
  name: my-gateway-config
  namespace: default
spec:
  extProc:
    kubernetes:
      resources:
        requests:
          cpu: "100m"
          memory: "128Mi"
        limits:
          cpu: "500m"
          memory: "512Mi"
      env:
        - name: OTEL_EXPORTER_OTLP_ENDPOINT
          value: "http://otel-collector:4317"

2. Reference from your Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: ai-gateway
  annotations:
    aigateway.envoyproxy.io/gateway-config: my-gateway-config

Migrating Endpoint Prefix Configuration

Before:

schema:
  name: OpenAI
  version: "/v1beta/openai"  # Deprecated

After:

schema:
  name: OpenAI
  prefix: "/v1beta/openai"

---

📦 Dependencies

Dependency	Version
Go	1.25.6
Envoy Gateway	v1.6
Envoy Proxy	v1.36.4
Gateway API	v1.4.0
Gateway API Inference Extension	v1.0.2

---

🙏 Acknowledgements

Special thanks to the growing community of adopters including Bloomberg, LY Corporation, Alan by Comma Soft, and NRP for their production insights, everyone who reported bugs, submitted PRs, and participated in design discussions, and the Envoy Gateway team for continued collaboration.

---

🔮 What's Next

• Additional provider integrations (AWS Bedrock InvokeModel, Gemini embeddings, Azure/AKS workload identity)
• Batch inference APIs for high-volume workloads
• Advanced caching strategies with prompt cache key and retention controls
• Upstream provider quota policies
• Sensitive data redaction for request and response bodies

v0.5.0-rc1

Release candidate for v0.5.0!

helm install aieg oci://registry-1.docker.io/envoyproxy/ai-gateway-helm --version v0.5.0-rc1 --namespace envoy-ai-gateway-system --create-namespace

v0.4.0

Envoy AI Gateway v0.4.0 - November 07, 2025

Release introducing Model Context Protocol (MCP) Gateway, OpenAI Image Generation, Anthropic support (direct and AWS Bedrock), guided output decoding for GCP Vertex AI/Gemini, cross-namespace references, enhanced authentication, and comprehensive observability improvements.

🔗 View release notes on the site

✨ New Features

Model Context Protocol (MCP) Gateway

New MCPRoute CRD

Introduces MCPRoute custom resource for routing MCP requests to backend MCP servers, enabling unified AI API for multiple MCP backends.

Complete MCP spec implementation

Includes streamable HTTP transport, JSON-RPC 2.0 support, and MCP spec-compliant OAuth 2.0 authorization with JWKS validation and Protected Resource Metadata.

Server multiplexing and tool routing

Aggregates multiple MCP servers behind a single endpoint with intelligent tool routing, tool filtering (exact match and regex patterns), and collision detection.

Upstream authentication

Supports both OAuth-based authentication and API key authentication for secure backend MCP server communication with configurable headers.

Session management

Implements MCP session handling with encryption, rotatable seeds, and graceful session lifecycle management.

Anthropic Provider Support

Direct api.anthropic.com support

Native integration with Anthropic's API at api.anthropic.com, complementing existing GCP Vertex AI Anthropic support.

AWS Bedrock native Anthropic Messages API

Support for Claude models on AWS Bedrock using the native Anthropic Messages API format instead of the generic Converse API, enabling full feature parity with direct Anthropic API including prompt caching and extended thinking.

Anthropic API key authentication

Native x-api-key header-based authentication matching Anthropic's API conventions and SDK patterns for direct Anthropic connections.

Passthrough translator with token usage tracking

Efficient passthrough translation layer that captures token usage and maintains API compatibility while minimizing overhead for both direct and AWS Bedrock Anthropic endpoints.

Standalone CLI auto-configuration

Auto-configuration from ANTHROPIC_API_KEY environment variable in standalone mode for zero-config deployments.

Guided Output Support for GCP Vertex AI/Gemini

Guided regex support

Constrains model outputs to match specific regular expressions for GCP Vertex AI/Gemini models, enabling structured text generation.

Guided choice support

Restricts model outputs to predefined choices for GCP Vertex AI/Gemini models, ensuring responses conform to expected values.

Guided JSON support

Ensures model outputs are valid JSON conforming to specified schemas for GCP Vertex AI/Gemini models, with OpenAI-compatible API translation.

Provider-Specific Enhancements

OpenAI Image Generation /v1/images/generations endpoint

End-to-end support for OpenAI's image generation API including request/response translation, Brotli encoding/decoding, and full protocol compatibility.

OpenAI legacy /v1/completions endpoint

Full pass-through support for OpenAI's legacy completions endpoint with complete tracing and metrics, ensuring backward compatibility.

Azure OpenAI embeddings support

Native support for Azure OpenAI embeddings API with proper protocol translation and token usage tracking.

AWS Bedrock reasoning tokens

Full support for reasoning/thinking tokens in AWS Bedrock responses for both streaming and non-streaming modes, properly exposing extended thinking processes in Claude models.

GCP Vertex AI safety settings

Support for GCP-specific safety settings configuration, allowing fine-grained control over content filtering and safety thresholds for Gemini models.

GCP Gemini streaming token accounting

Accurate completion_tokens reporting in streaming usage chunks for Gemini models, ensuring proper token accounting during streaming responses.

Cross-Namespace Resource References

Cross-namespace AIServiceBackend references

AIGatewayRoute can now reference AIServiceBackend resources in different namespaces, enabling multi-tenant and organizational separation patterns.

ReferenceGrant validation

Comprehensive ReferenceGrant integration following Gateway API patterns, with automatic validation and clear error messages when grants are missing.

Enhanced Upstream Authentication

AWS SDK default credential chain

Support for AWS SDK's default credential chain including IRSA (IAM Roles for Service Accounts), EKS Pod Identity, EC2 Instance Profiles, and environment variables, eliminating need for static credentials or OIDC settings

Azure API key authentication

Native Azure OpenAI API key authentication using the api-key header, matching Azure SDK conventions and console practices.

Traffic Management and Configuration

Header mutations at route and backend levels

New headerMutation fields in both AIServiceBackend and AIGatewayRouteRuleBackendRef enable header manipulation with smart merge logic for advanced routing scenarios.

InferencePool v1 support

Updated to Gateway API Inference Extension v1.0, providing stable intelligent endpoint selection with enhanced performance and reliability.

Cached token usage tracking for actual token usage reporting

Captures and reports cached token statistics from cloud providers (Anthropic, Bedrock, etc.), providing accurate cost attribution for prompt caching features.

Standalone Mode and CLI

Docker image support

Official Docker images for the aigw CLI published to GitHub Container Registry, enabling containerized standalone deployments with proper health checks and lifecycle management.

Multi-provider auto-configuration

Zero-config standalone mode with automatic configuration from OPENAI_API_KEY, AZURE_OPENAI_API_KEY, or ANTHROPIC_API_KEY environment variables. Generates complete Envoy configuration with OpenAI SDK compatibility.

MCP server configuration

Native MCP support in standalone mode via --mcp-config and --mcp-json flags, enabling unified LLM and MCP server configuration in a single aigw run invocation without Kubernetes.

XDG Base Directory standards

Proper separation of configuration, data, state, and runtime files following XDG Base Directory specification, improving organization and enabling better cleanup and management of aigw state.

Enhanced readiness monitoring

Improved Envoy readiness detection and status reporting in standalone mode, providing clear insights into when the gateway is ready to accept traffic with better error messages.

Consolidated admin server

Unified admin server on a single port serving both /metrics and /health endpoints, simplifying monitoring and health check configuration.

Improved error handling

aigw CLI now fails fast and exits cleanly if external processor fails to start, preventing silent failures and improving debugging experience.

Type-safe Kubernetes client SDK

Generated client libraries for all AI Gateway CRDs following standard Kubernetes client-go patterns, enabling developers to build controllers, operators, and custom integrations with type safety.

Observability Enhancements

MCP operations observability

Comprehensive monitoring, logging, and tracing for MCP operations with configurable access logs and metrics enrichment for MCP server interactions and tool routing.

Image generation tracing and metrics

OpenInference-compliant distributed tracing and OpenTelemetry Gen AI metrics for image generation requests with detailed request parameters and timing information.

OpenTelemetry native metrics export

Support for OTEL-native metrics export (in addition to Prometheus), enabling integration with Elastic Stack, OTEL-TUI, and other OTEL-native observability systems. Includes console exporter for ad-hoc debugging.

Embeddings tracing implementation

Complete OpenInference-compliant tracing for embeddings operations, complementing existing chat completion tracing.

Enhanced /messages endpoint metrics

Distinct metrics for Anthropic's /messages endpoint, providing accurate attribution separate from /chat/completions endpoints.

Original model tracking

Metrics now track both the original requested model and any overridden model names, providing accurate attribution in multi-provider and model virtualization scenarios.

🔗 API Updates

• New MCPRoute CRD
  • Introduces MCPRoute custom resource with comprehensive fields for MCP server configuration, tool filtering, authentication policies (OAuth and API key), and Protected Resource Metadata.
• Cross-namespace references in AIGatewayRoute
  • Added namespace field to AIGatewayRouteRuleBackendRef, enabling cross-namespace backend references with ReferenceGrant validation.
• Header mutations at route and backend levels
  • Added headerMutation fields to both AIServiceBackend and AIGatewayRouteRuleBackendRef for backend-level and per-route header manipulation with smart merge logic.
• New AWSAnthropic API schema
  • Added AWSAnthropic schema for Claude models on AWS Bedrock using the native Anthropic Messages API format, providing full feature parity with direct Anthropic API.
• Anthropic API key authentication
  • Added AnthropicAPIKey to BackendSecurityPolicy for x-api-key header authentication.
• Azure API key authentication
  • Added AzureAPIKey to BackendSecurityPolicy for api-key header authentication.
• **AWS credential chain support...

v0.4.0-rc2

Release candidate for v0.4.0!

helm install aieg oci://registry-1.docker.io/envoyproxy/ai-gateway-helm --version v0.4.0-rc2 --namespace envoy-ai-gateway-system --create-namespace