Release v1.9.0 — Per-user MCP sign-in, tags from JWT claims, upstream errors as text · api7/aisix

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

📖 Documentation — https://docs.api7.ai/ai-gateway/
⚡ Gateway quickstart (one container, declarative resources.yaml) — https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:1.9.0

The rolling latest tag and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX 1.9.0 lets each user sign in to upstream MCP servers from their MCP client, and turns claims of a verified token into cost-attribution tags. Upstream failures now reach callers as the upstream's own text with its code and param, and failure records say what went wrong.

Per-user OAuth to upstream MCP servers (#1394, #1399, #1400, #1402): each user signs in to the upstream from their MCP client, with or without a gateway credential, and the gateway calls the upstream with that user's token.
Cost-attribution tags from JWT claims (#1407): oidc_providers.tag_claims lets tags follow the identity provider instead of being maintained on every API key.
Upstream errors read as text on every route (#1410, #1413): error.message is the upstream's text, and a 4xx carries code and param. This changes the error contract; see Breaking changes.
Failure records name their failure (#1397, #1398), and under content_mode: full they carry the request and response bodies on every endpoint and status.
A refused realtime session answers and records as a failure (#1410, #1411, #1412, #1413), with the upstream's reason in the client's error frame.
Vertex AI Gemini returns reasoning and generated images (#1391, #1393).

Control-plane, console and AISIX Enterprise / AISIX Cloud changes in this release (administering connected MCP users, provider_key budget accounting, platform cost, the Logs page, control-plane API changes) are described in the AISIX release notes.

⚠️ Breaking changes

resources.yaml has no format break in this release: existing files load as they are. The changes below are in gateway behaviour, and each says what to update.

Upstream error responses carry text, `code` and `param` (#1410, #1413)

On an upstream failure, error.message now holds the upstream's text rather than its whole error body as a JSON string. This applies to /v1/rerank (including Cohere), /v1/audio/speech, /v1/audio/transcriptions, /v1/audio/translations, /v1/images/edits, and native /v1/responses (including streaming). A 4xx now also carries the upstream's code and param. A 5xx carries neither. Status codes do not change.

Before:

{"error":{"message":"{\"error\":{\"message\":\"bad\",\"type\":\"invalid_request_error\",\"code\":\"bad_param\",\"param\":\"x\"}}","type":"upstream_error"}}

After:

{"error":{"message":"bad","type":"upstream_error","param":"x","code":"bad_param"}}

Before upgrading, change any caller that parses JSON out of error.message on these routes. Read error.message as text, and error.code and error.param as fields. The access log's error and the failure row's error_message read upstream returned HTTP 500: bad, and the full-content response field still holds the upstream's body as sent.

Related changes:

Other error body shapes, on every route. A body shaped {"message":"x"}, {"error":"x"}, {"detail":"x"}, or {"detail":{"message":"x"}} now reads as x, including on chat, completions, embeddings, images, messages, count tokens, and bridged responses, which used to return the raw body text. A blank field no longer hides the text of a later one. A non-JSON body stays the text as sent.
/v1/decisions. A 4xx now carries the upstream's code and param. An error body past 64 KiB is no longer parsed, so the caller receives its first 1 KiB as text.
Azure OpenAI keys. On /v1/chat/completions and /v1/responses, a missing deployment now answers code: model_not_found, where it answered no code. A 4xx now carries param, and a numeric error.code now answers as a string. The status, error.type, and error.message do not change. Audio and realtime still carry Azure's own code untranslated, and /v1/messages still omits code.
/v1/videos. On POST /v1/videos, GET /v1/videos/{id}, and GET /v1/videos/{id}/content, for every video provider:
- error.message is the upstream's text, without the code: prefix. A client that parsed the code out of message must read error.code.
- A 4xx carries the upstream's code and param, and a DashScope top-level string code. error.type stays upstream_error. A 5xx carries text only.
- The literal upstream error is gone. A non-JSON body, or JSON with no text field, is returned as text. An empty body or a blank message gets the status fallback, such as upstream returned 400 Bad Request.
- An upstream's Retry-After now reaches the caller on a 429. On any status that trips the model's cooldown, it also sets the cooldown's length, and it paces same-target retries. This includes a poll or content request, so a 429 on a poll can hold the model's new submits for up to the cooldown's max_seconds.
- An error body past 64 KiB is no longer parsed, so the caller receives its first 1 KiB as text.
- A non-2xx whose body cannot be read whole is now decided by its status, where it used to be a 502 upstream_decode_error that was neither retried nor counted toward cooldown. The message is the status fallback, a 4xx keeps its status, a 5xx answers 502, a 429 keeps Retry-After, and retries and cooldown follow that status.
- The failure row and access log cap a message at 1 KiB, and a body with a code but no text no longer reports the code.

The /v1/files, /v1/batches, and fine-tuning routes and passthrough routes still relay a non-2xx as the upstream sent it.

Inject passthrough routes remove every credential header (#1405)

A passthrough route with credential_mode: inject now removes every inbound credential header before forwarding: authorization, x-api-key, api-key, x-goog-api-key, cookie, proxy-authorization, x-amz-security-token, x-amz-date, and x-amz-content-sha256. The ProviderKey's strip_headers no longer decides this, and a glob such as x-* or * does not bring one back.

Before upgrading, name each header that a route must still forward, exactly, in forward_client_headers. This includes a ProviderKey that un-stripped cookie. If the named header is the one the route injects into, the upstream receives the caller's value instead of the injected one. credential_mode: forward_client is unchanged.

Failure records carry real text and more content (#1397, #1398)

Every failure row now names its failure in error_message, at any content_mode. The text can echo what a caller sent or what an upstream answered, and it reaches OTLP and SLS, including under metadata_only. A value that a monitor-mode guardrail matched, or one quoted by a validation error raised before guardrails run, can appear there when an upstream or a parser echoes it. A guardrail's own refusal still does not carry what matched. An upstream failure reads upstream returned HTTP <status>: <upstream text>.

Under content_mode: full, every failure row now carries the request body and its own response body, on every endpoint and status:

Exports now include request bodies for upstream 401 and 403 responses and for gateway refusals such as a forbidden model.
A row that fails before guardrails finish carries an unmasked request body.
Every fallback attempt repeats the request body, so export volume grows on failing traffic.
Under a full-content exporter, the gateway reads an A2A agent's error body before answering, so a slow agent can delay the answer up to the agent's timeout.

Before upgrading, review who can read your exports, and how much export volume you can absorb.

Vertex AI Gemini `max` and `xhigh` effort can return 400 (#1391)

Callers that ask for thinking on Vertex AI Gemini 2.5 or later now receive reasoning on their protocol's reasoning field. A stream's first event can be reasoning rather than text.

Gemini has no setting for an effort of max or xhigh. On a model that does not think by default, such as gemini-2.5-flash-lite, gemini-2.5-flash-image, or gemini-3.5-transcribe, the upstream answers 400 Thinking_config.include_thoughts is only enabled when thinking is enabled. 1.8.0 returned 200. The gateway passes that error through, so choose a supported effort for those models.

Output guardrails do not scan thought text. A deployment with an output guardrail on a Vertex AI Gemini model now passes this text unscanned to callers who asked for it.

Upstream 200 responses that report an error become failures (#1389)

A 2xx response or stream frame with a top-level error that carries something is now an in-band upstream failure on the covered paths. The caller receives 502, or the embedded 4xx, with upstream_in_band_error.

Completions and image generation used to return this as a 200 carrying the error object.
Native /v1/responses used to do the same for a bare error envelope. A response object with status: "failed" is still relayed as an answer.
/v1/messages/count_tokens no longer relays such a body as a count.
Other typed paths used to return 502 upstream_decode.
Passthrough routes still relay the upstream's bytes and status, and now record the failure.
/v1/realtime follows its own rule, described under Upgrade notes.
null, {}, and "" in error are not failures.

Exported content changes shape (#1394, #1395, #1396, #1398)

Consumers of full-content OTLP, SLS, and Datadog records should account for these changes:

Record	Change
Chat prompts	Carry every block the client sent. Multi-block text is several text parts rather than one concatenated part.
Base64 media	Small payloads no longer stay verbatim. Recognized media is replaced with `[aisix: base64 data uri omitted, N bytes]` in prompts, MCP `tools/call` arguments and results, and passthrough bodies. A data URI keeps its mime-type head.
Image endpoints	`b64_json` is replaced, so the response field is no longer the full response JSON.
MCP tool results	A result that used to overflow into `aisix.content.response` because of an image now stays whole under `gen_ai.tool.call.result`.
Vertex AI Gemini `provider_request_id`	Now holds `responseId` instead of being empty.
Capability `501` rows	Gain `error_class: config`, and the OTLP server span reports `error.type: config` rather than `501`.
`endpoint` metric label	Unserved paths under `/mcp/{server}/` report `other` rather than `/mcp/{server}`.

Callers still receive their media. Object storage carries no content and is unchanged.

🧩 MCP

Per-user OAuth to upstream servers (#1394, #1399, #1400, #1402). type: mcp servers gain auth_type: oauth2_authorization_code, with optional authorization_url and registration_url beside the existing client_id, secret, token_url and scopes. Endpoints that are not configured are discovered from the upstream. Upstream tokens never reach the client, and the client's Authorization never reaches the upstream.
- Sign-in without a gateway credential. With exactly one enabled oidc_providers entry carrying the new login: {client_id, client_secret, scopes}, a client can connect with only the server's URL. Register /mcp-oauth/login/callback at the provider. With none, the client brings a gateway credential in x-aisix-api-key. A second enabled provider with login, or login on a provider with hmac_secret or no issuer, makes sign-in answer 500 mcp_oauth_misconfigured, and so does /mcp/{server} without mcp_auth_settings.resource_url.
- Storage. A gateway with no data-plane manager keeps credentials and sessions in memory: one process, lost on restart, so replicas do not share them and users sign in again after a restart.
- What sends users back through sign-in. Changing a server's url or token_url, or a provider's issuer or identity_claim, leaves stored upstream credentials out of reach.
- Network. The proxy listener serves new paths under /.well-known/, /mcp/{server}/, and /mcp-oauth/, each with its own endpoint metric label. A front proxy must route them. outbound.ssrf_protection covers OAuth documents and endpoints that an upstream names on another authority.
- Export and rollback. aisix export writes a placeholder for login.client_secret. A gateway older than 1.9.0 cannot load a server with this auth_type, and it ignores login.
Path-prefixed resource URLs (#1392). mcp_auth_settings.resource_url can carry a path prefix, such as https://host/aisix/mcp, for gateways behind a front proxy that routes by prefix and strips it. The path must end in /mcp at a segment boundary. A plain /mcp URL behaves as before.

🔐 Authentication

Cost-attribution tags from JWT claims (#1407). An OIDC provider can map claims of the verified token to tags, so usage events, tag budgets, and tag: rate-limit conditions read them. Nothing is written back to the key.
```
tag_claims:
  department: department
  subsidiary: org.subsidiary
```
- A claim overrides the same tag key with the precedence claim, key, team, member.
- Strings, numbers, and arrays of them become text. Arrays are sorted, deduplicated, and joined with ,. An invalid value yields nothing, and the key keeps what its other layers give it.
- When several tokens share an API key, the key-labelled aisix_budget_* gauges reflect the most recent request's verdict. A batch attribution row carries the tags of the caller that observed the batch complete.

📊 Observability

is_final_attempt on usage records (#1397). Of all the rows sharing a request_id, the last is true, so a query can show one row per request. It is always written on SLS and object storage, and as aisix.is_final_attempt on OTLP. Filtered rows do not total the tokens or cost of hidden attempts, so do not use them to compute spend.
error_source on request counters (#1397). aisix_llm_requests_total, aisix_proxy_requests_total, and aisix_proxy_failed_requests_total gain a default error_source label: upstream, aisix, or empty for a success or a caller that left. {error_source!="aisix"} gives call volume without the gateway's own refusals, which status codes cannot do because both sides answer 429. A stream a guardrail refused after a 200 head reads status="200", error_source="aisix". A metric label selection that already lists labels for these families does not pick it up, so add it explicitly. The label at most doubles failure series on statuses both sides answer.
Failure diagnostics (#1397, #1398, #1410). Empty upstream errors now read upstream returned HTTP <status> with no error detail in the response, and every failure row and access-log line at 400 or above names its failure. Passthrough and job rows for a relayed non-2xx gain error_class: upstream_status. An in-band failure now reads upstream reported an in-band error: <message> in error_message and the access log's error, on streaming and non-streaming requests alike, where the access log said in-band stream error. A stream's closing error frame keeps its text.
Gateway logs (#1406). Outbound failures now log the underlying cause, such as an untrusted certificate, instead of only error sending request.

🔌 Providers and API surface

Vertex AI: Gemini generated images on chat completions (#1393). /v1/chat/completions now returns Gemini image output in message.images and delta.images, and a client's modalities becomes generationConfig.responseModalities. Image models can return an interim thought image before the final one, so read the last entry for the final image.
- /v1/messages and /v1/responses still discard images, so an image-only answer is an empty 200 on those two endpoints.
- Output guardrails do not scan or block on images.
- Image-related Gemini finish reasons now finish as content_filter.
- A response with images, from any upstream, is not stored in the response cache.
- Generated image data stays out of usage events, access logs, and exporters.
- Assistant images in history are written back to Gemini as inlineData. An entry that cannot be written, or an http:// URL there, is a 400 naming messages[i].images[j].
- gemini-nano-banana-2.1 sends thought text even when the client did not ask for thinking, so such a client can see reasoning_content.
Vertex AI Gemini and Bedrock Converse response ids (#1395). Chat and non-streaming Messages responses no longer return an empty id.
Gateway memory (#1387). Native /v1/responses, /v1/messages, /v1/messages/count_tokens, /v1/rerank, and JSON /v1/audio/speech requests whose only rewrite is the model name use about one body-size less peak memory when content capture is off.

🐛 Fixes

An upstream that answers a failure status, sends part of the body, and then goes silent no longer holds a streaming request with no limit (#1410). This covers streaming /v1/chat/completions, /v1/messages, /v1/responses, speech, and transcriptions, on the OpenAI, Anthropic, Azure, Vertex AI, and Bedrock Chat Completions paths. The read now gets the model's stream budget, after which the caller receives the status fallback, such as 502 upstream returned 500 Internal Server Error, and the row reads failed to read response body: timed out after <n>ms. The upstream's status and Retry-After still drive retries and cooldown. A model with timeout: 0 has no stream budget, so its read stays unbounded.
The A2A agent card no longer builds a broken URL from a chained X-Forwarded-Proto such as https,http (#1410). The first hop is read, in any case, and a value other than http or https is ignored.

⬆️ Upgrade notes

These notes apply when you upgrade from 1.8.x or earlier, in addition to Breaking changes above.

Realtime: a refused session answers and records as a failure (#1410, #1411, #1412, #1413)

A /v1/realtime session the upstream never let start now tells the client why and records as a failure, whichever way it was refused. No configuration needs editing. A client that matches the sentence failed to connect to the upstream realtime endpoint for a refused handshake must stop doing so. Alerts and dashboards that count realtime dial hangs under 502, OpenAI wrong-key sessions under 200, or refused handshakes under 502 will see these sessions move.

The WebSocket upgrade to the client still succeeds before the dial. Apart from the last row, the client's error frame is followed by close 1011 with reason upstream connect failed, as before.

What happened	Client's `error` frame	Recorded status and class
Handshake refused with a 4xx	The upstream's text, with its `code` and `param`	The upstream's status, so `401` has outcome `client_error` and `429` has `rate_limited`; `upstream_status`
Handshake refused with a 5xx	The upstream's text only	`502`; `upstream_status`. The recorded text names the real status.
Handshake answered with a 2xx or 3xx that is no upgrade, or a status past 599	The body's text, or a fallback such as `upstream returned 200 OK`	`502`; `upstream_status`
Handshake refused with an empty or unreadable body	A status fallback such as `upstream returned 500 Internal Server Error`	As for its status. The text reads `upstream returned HTTP <status> with no error detail in the response`, or `upstream returned HTTP <status>: failed to read response body: <reason>`.
Dial past `upstream.connect_timeout_ms`	`failed to connect to the upstream realtime endpoint`	`504`; `timeout`, with text `upstream request timed out after <n>ms: ...`
No answer at all, such as a transport, DNS, or TLS failure	`failed to connect to the upstream realtime endpoint`	`502`; `transport`
Upgrade accepted, then an `error` event before any other frame	None from the gateway. The client gets the upstream's frames and its close code. A dropped connection with no Close still closes `1011`.	From the event, below; `upstream_in_band`, with text `upstream reported an in-band error: <upstream text>`

The status of the last row comes from the event: invalid_api_key is 401; rate_limit_exceeded and insufficient_quota are 429; invalid_model and model_not_found are 404; any other invalid_request_error is 400. A numeric code in 400-499 is used as itself, one in 500-599 is 502, and everything else is 502. These statuses apply to status_code on the row, the access log, and the status and outcome labels on the request metrics.

Cooldown. A handshake refused with a status, or a refused session with a status, now takes trigger_statuses and honor_retry_after rather than trigger_on_transport. With the default list, an OpenAI wrong-key session now cools the model down, and a 400 or an unknown model does not. A dial timeout now takes trigger_on_timeout, so trigger_on_transport: false no longer keeps the model available on a dial hang. A refused session with no status takes trigger_on_transport.
aisix_deployment_*. The success sample now waits for the first frame that is not an error event, or for the end of the session, rather than being taken at the 101. A refused session counts one failure and no success. A session that an upstream failure ended before any frame now counts as a failure.
Failure rows. On a refused handshake, response at content_mode: full is the upstream's body as sent, upstream_response_headers carries the exporter's listed headers, provider_request_id is read from the response's request-id headers, upstream_latency_ms is the dial's duration, and OTLP spans carry the server address. On a refused session, response at content_mode: full holds the error frame as sent.
Handshake body. The gateway now reads a refused handshake's whole body, up to 64 KiB and within the model's stream budget, so the frame no longer depends on the body's size and timing. A stalled body holds the client's frame for up to one stream budget past the dial, and with no budget for as long as the upstream stalls and the client stays.

A session that became usable is unchanged. An error event after its first usable frame, such as one OpenAI sends for a bad client event while the session stays open, does not make it a failure.

Output guardrails run on responses with no text (#1393)

The output guardrail chain now also runs on responses with no text on the chat stream hold-back, both /v1/messages streams, the /v1/responses bridge, the end-of-stream monitor scan, and buffered transcripts. Text-matching kinds match nothing and let such a response through.

A custom script decides for itself. One that blocks unconditionally now also refuses an empty or image-only stream and an empty transcript. ctx.segments can be empty, and a script that throws on a fail-closed row refuses the response. Review scripts that read ctx.segments[0].

Dotted claim names read literally first (#1407)

identity_claim, bound_claims, claim mapping match[].claim, and the new tag_claims now read the whole string as one top-level claim name first, and only then traverse dots. A name such as https://acme.com/dept, which none of them could read before, now resolves. A token carrying both a literal a.b claim and a nested a: {b} now resolves a.b to the literal one. No configuration needs editing.

What's Changed

ci(e2e): commit the harness pnpm lockfile and install it frozen by @jarvis9443 in #1382
test(e2e): stop the heap pprof symbolization case from flaking by @jarvis9443 in #1384
ci(ghcr-retention): retry GitHub API 5xx and treat DELETE 404 as done by @jarvis9443 in #1385
fix(providers): treat an upstream 200 whose body reports an error as an in-band failure by @jarvis9443 in #1389
fix(vertex): carry Gemini thoughts as reasoning on every client protocol by @jarvis9443 in #1391
feat(mcp): allow a path prefix in mcp_auth_settings.resource_url by @jarvis9443 in #1392
feat(vertex): return Gemini generated images on chat completions by @jarvis9443 in #1393
fix(telemetry): keep media parts without base64 in full-content exports; never return an empty response id by @jarvis9443 in #1395
fix(telemetry): omit base64 media from MCP tool results in full-content exports by @jarvis9443 in #1396
feat(telemetry): failure rows name their failure and carry both bodies; request counters gain error_source; rows mark the final attempt by @jarvis9443 in #1397
feat(mcp): per-user OAuth to upstream MCP servers, with keyless interactive sign-in by @jarvis9443 in #1394
fix(telemetry): every failure row and access-log line names its failure by @jarvis9443 in #1398
fix(mcp): bind gateway MCP sessions to the identity provider that issued them by @jarvis9443 in #1399
fix(mcp): key stored upstream credentials by identity claim; sign-in and refresh read the store by @jarvis9443 in #1400
fix(mcp): wait for another gateway's proactive refresh instead of returning the token it replaces by @jarvis9443 in #1402
perf(proxy): avoid redundant request body copies on native handlers by @xz-dev in #1387
fix(passthrough): keep every credential slot home on inject routes by @jarvis9443 in #1405
fix(obs): log the underlying cause of outbound HTTP failures by @jarvis9443 in #1406
feat(auth): derive cost-attribution tags from verified JWT claims by @jarvis9443 in #1407
docs(claude): state how a stored field's rename is carried and retired by @jarvis9443 in #1408
fix(proxy): read an upstream's error text on routes that send their own request by @jarvis9443 in #1410
fix(realtime): answer a refused handshake as an HTTP route would by @jarvis9443 in #1411
fix(realtime): record a session the upstream never let start as a failure by @jarvis9443 in #1412
fix: answer video failures the way chat does, translate Azure 4xx without a Content-Type, record realtime model errors as 404 by @jarvis9443 in #1413
ci(image): add swap for the PGO build so the arm64 leg stops dying in phase A by @jarvis9443 in #1416

Full Changelog: v1.8.0...v1.9.0

Navigation Menu

Repository navigation

v1.9.0 — Per-user MCP sign-in, tags from JWT claims, upstream errors as text

Choose a tag to compare

Sorry, something went wrong.

Sorry, something went wrong.

Uh oh!

No results found

🚀 Get started

📦 Download

⚠️ Breaking changes

Upstream error responses carry text, `code` and `param` (#1410, #1413)

Inject passthrough routes remove every credential header (#1405)

Failure records carry real text and more content (#1397, #1398)

Vertex AI Gemini `max` and `xhigh` effort can return 400 (#1391)

Upstream 200 responses that report an error become failures (#1389)

Exported content changes shape (#1394, #1395, #1396, #1398)

🧩 MCP

🔐 Authentication

📊 Observability

🔌 Providers and API surface

🐛 Fixes

⬆️ Upgrade notes

Realtime: a refused session answers and records as a failure (#1410, #1411, #1412, #1413)

Output guardrails run on responses with no text (#1393)

Dotted claim names read literally first (#1407)

What's Changed

Contributors

Uh oh!

v1.9.0 — Per-user MCP sign-in, tags from JWT claims, upstream errors as text

🚀 Get started

📦 Download

⚠️ Breaking changes

Upstream error responses carry text, code and param (#1410, #1413)

Inject passthrough routes remove every credential header (#1405)

Failure records carry real text and more content (#1397, #1398)

Vertex AI Gemini max and xhigh effort can return 400 (#1391)

Upstream 200 responses that report an error become failures (#1389)

Exported content changes shape (#1394, #1395, #1396, #1398)

🧩 MCP

🔐 Authentication

📊 Observability

🔌 Providers and API surface

🐛 Fixes

⬆️ Upgrade notes

Realtime: a refused session answers and records as a failure (#1410, #1411, #1412, #1413)

Output guardrails run on responses with no text (#1393)

Dotted claim names read literally first (#1407)

What's Changed

Contributors

Uh oh!

Upstream error responses carry text, `code` and `param` (#1410, #1413)

Vertex AI Gemini `max` and `xhigh` effort can return 400 (#1391)