Skip to content

v1.8.0 — MCP on the caller's behalf, PROXY protocol, gateway-side budgets

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Oct 15:44
· 18 commits to main since this release
v1.8.0
248ed00

true
v1.8.0

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:1.8.0

The rolling latest tag and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX 1.8.0 lets MCP servers act on the caller's behalf, accepts PROXY protocol headers from trusted load balancers, and moves budget decisions onto the gateway itself, so a control-plane outage no longer turns into a blanket refusal. An idle gateway now needs a fraction of the memory it used to.

  • MCP servers can act on the caller's behalf (#1361, #1362): relay the caller's own token upstream, or exchange it for a downstream token through the Microsoft Entra ID On-Behalf-Of grant. A new x-aisix-api-key header carries the gateway key so Authorization stays free for the caller's own token.
  • PROXY protocol v1 and v2 from trusted load balancers (#1374), on every proxy listener, detected automatically.
  • Budgets are decided on the gateway (#1366) from budget state the control plane pushes with the configuration. A control-plane or database outage no longer refuses traffic.
  • Every proxy route emits every per-request metric family that applies to it (#1367), and access logs carry token counts on every endpoint (#1365).
  • Unknown startup-config keys are ignored with a warning instead of stopping the gateway (#1368).
  • Idle memory drops from about 120 MiB to under 10 MiB (#1370).

Control-plane, console and AISIX Enterprise / AISIX Cloud changes in this release (two-factor sign-in, password policy, control-plane API and Helm aisix-cp changes) are described in the AISIX release notes.

⚠️ Breaking changes

See Upgrade notes for what to update.

  • Debug response headers on /v1/chat/completions are off by default (#1364). x-aisix-call-id, x-aisix-served-by, x-aisix-route, x-aisix-cache, x-aisix-cache-layer and x-aisix-cache-similarity are no longer sent unless proxy.expose_debug_headers: true is set in the gateway config (Helm chart aisix: config.proxy.expose_debug_headers). The same facts are in usage events, access logs, metrics and traces. x-aisix-request-id, server, x-ratelimit-* and retry-after are still sent.
  • Four access-log fields take their usage-event names (#1365): status → status_code, error_kind → error_class, model → requested_model, latency_ms → downstream_latency_ms. Only the new names are written.
  • aisix_llm_spend_micro_usd_total is removed (#1367). Only Realtime ever put a value on it. Read spend from cost_usd on usage events, or the control plane's recorded cost. A config that still lists it under observability.metrics.labels boots and warns.
  • One inbound credential rule on every entry (#1383). An empty x-aisix-api-key, Authorization or x-api-key header counts as an offered credential and is a 401; it is no longer skipped, so an MCP client using anonymous access that sends one of them empty is now refused instead of served anonymously. On gateway_key passthrough routes, an Authorization that is present but not a usable Bearer credential no longer falls back to x-api-key; it gets 401.
  • A connection whose first bytes are PROXY is read as a PROXY v1 header (#1374) and closed unless the peer is in proxy.real_ip.trusted_proxies. It used to get an HTTP 400.
  • Dual-stack ([::]) listeners report and match IPv4 clients as plain IPv4 (#1374) instead of ::ffff:a.b.c.d, in allowlists, usage events, OTLP client.address and logs.

🧩 MCP and gateway credentials

  • New gateway credential header x-aisix-api-key (#1361, #1383). When present it takes precedence over Authorization and x-api-key for gateway authentication, so a caller can send its gateway key there and keep Authorization for its own upstream token. It is accepted on every /v1/* endpoint, /mcp, /mcp/{server}, /a2a, the /v1/realtime handshake and gateway_key passthrough routes, and is never forwarded upstream. An empty value is a 401, with no fall-through to the next header.
  • Inbound credentials follow one rule (#1383). bearer <key> and BEARER <key> are accepted on every entry (the key itself is case-sensitive), and a 401 for an empty or malformed x-aisix-api-key or x-api-key now names that header instead of Authorization.
  • New MCP server auth type oauth_delegate (#1361), for mcp and openapi servers: the gateway relays the caller's own Authorization to the upstream, but only when that header was not the gateway credential (the gateway key came in x-aisix-api-key). Only servers the caller's MCP access rules reach receive it.
  • New MCP server auth type oauth2_token_exchange with token_exchange_profile: entra_obo (#1361, #1362): the gateway exchanges the caller's token, verified by a JWT trust provider, through the Microsoft Entra ID On-Behalf-Of grant and sends only the minted token upstream. Tokens are cached until shortly before expiry and re-exchanged once on an upstream 401; the exchange is bounded by the server's timeout_ms. On /mcp/{server} failures have distinct statuses (401 token_exchange_subject_missing / token_exchange_subject_rejected, 500 token_exchange_misconfigured, 503 token_exchange_unavailable), and an Entra Conditional Access step-up reaches the client as an insufficient_claims challenge. On the aggregated /mcp, a failing server only drops its own tools.

💰 Budgets

  • Hard-stop budgets are decided on the gateway (#1366) from budget state the control plane pushes with the configuration; the gateway no longer asks the control plane on each request. Raising or lowering a limit takes effect immediately, a block lifts at the period boundary even while the control plane is unreachable, and a control-plane or database outage no longer refuses traffic after about 10 minutes.
  • provider_key budgets act per upstream credential (#1366): an exhausted provider key removes only the routing targets that use it, so a Model Group with another target keeps serving. It does not consume a fallback slot. When every candidate is excluded the caller still gets 429 budget_exceeded. MCP, A2A and gateway-internal calls are not subject to it.
  • Budget enforcement is sticky or closed (#1366). sticky (default) keeps enforcing the last pushed state through an outage. closed (strict) refuses, once usage reporting has not reached the control plane for the grace period (budget_closed_grace_seconds, default 600 s, 60–86400), only the keys and upstreams a budget covers, with 503 budget_unverified; everything else keeps serving, and the next successful report lifts it.
  • New metric aisix_usage_delivery_unhealthy_seconds (#1366): how long usage events have been waiting without reaching the control plane.
  • The etcd watch uses HTTP/2 keepalive (#1366), so a silently dropped connection is detected and recovered in about 20 s instead of missing configuration updates indefinitely.

📊 Observability

  • Every client-facing proxy route emits every per-request metric family that applies to it (#1367): aisix_request_e2e_latency_seconds on all routes except /v1/realtime, time to first token on streamed audio, streamed /a2a and opaque passthrough streams, token families on /a2a, deployment families on video follow-ups, realtime and files / batches / fine-tuning, rate-limit remaining gauges at the quota gate, and 401 / 403 counted on every route including /mcp.
  • Access logs carry token counts on every endpoint (#1365): prompt_tokens / completion_tokens / total_tokens on non-streamed /v1/messages, /v1/embeddings, /v1/responses, /v1/completions, /v1/rerank, images, audio, /v1/decisions and /a2a lines, and the /v1/realtime session-close line carries the session totals.
  • Realtime usage events record error_class (transport, timeout) when the upstream ends a session (#1365).

⚙️ Deployment and configuration

  • PROXY protocol v1 and v2 on every proxy listener, plaintext and TLS (#1374). A header is honoured only from a socket peer in proxy.real_ip.trusted_proxies; from any other peer the connection is closed with a WARN. Detection is automatic, connections without a header keep working on the same port, and the address the header carries is what usage events, access logs, allowed_cidrs, source allowlists, MCP source_cidrs and client-IP hash routing see. No new config key.
  • IPv4 CIDRs match IPv4 clients on [::] listeners (#1374) in trusted_proxies, allowed_cidrs, source allowlists and MCP source_cidrs.
  • The startup config no longer refuses keys it does not know (#1368): a misspelled or retired key is ignored and named in a startup WARN (written to stderr when the log level hides warnings). A config that fails to load for another reason also lists the ignored keys in its error.
  • Resource-schema validation memory drops from about 115 MiB to about 4 MiB per gateway process (#1370); an idle gateway now sits under 10 MiB of allocated memory.
  • The TLS library is updated to a patched rustls release (#1376).

⬆️ Upgrade notes

These notes apply when you upgrade from 1.7.1 or earlier.

  • Turn debug headers back on if you read them. Set proxy.expose_debug_headers: true, or read the same facts from usage events, access logs, metrics or traces.
  • Update access-log parsers. Anything that parses the proxy request completed line by field name must use status_code, error_class, requested_model and downstream_latency_ms.
  • Stop reading aisix_llm_spend_micro_usd_total, and remove it from observability.metrics.labels.
  • Metric labels change on a few routes. The two /.well-known/oauth-protected-resource routes get their own endpoint label instead of other, and A2A agent-card fetches move from endpoint="/a2a" to /a2a/{agent}/.well-known/agent-card.json, so they no longer count as agent calls. aisix_component_entries{component="budget_cache"} is no longer reported.
  • Check the startup WARN lines after editing the config. A misspelled key no longer stops the gateway; it is ignored. The retired observability.metrics.otlp, observability.tracing and observability.usage_event blocks are removed: a config that still carries them boots and warns, so delete them. AISIX_DP_BUDGET_STALE_MAX_SECONDS is gone as well; a gateway that still sets it boots and warns.
  • Rewrite ::ffff:a.b.c.d allowlist entries on [::] listeners as plain IPv4 (trusted_proxies, allowed_cidrs, passthrough source allowlists, MCP source_cidrs), and expect client IPs in usage events, OTLP client.address and logs in the IPv4 form.
  • Behind an L4 load balancer that sends PROXY headers, add the balancer's addresses to proxy.real_ip.trusted_proxies.
  • MCP clients using anonymous access must not send an empty x-aisix-api-key, Authorization or x-api-key header; it is now a 401.
  • On gateway_key passthrough routes, send the gateway key in x-aisix-api-key and keep Authorization for the upstream. A request whose Authorization is present but not a usable Bearer credential no longer falls back to x-api-key.
  • Budget enforcement mode. Organizations whose mode was open are switched to sticky; open is no longer accepted. In sticky mode, a 1.7.1 gateway answered every request with 429 once the control plane had been unreachable for more than 600 s; a 1.8.0 gateway blocks only requests whose budget is already exhausted. In closed mode, after the grace period (budget_closed_grace_seconds, default 600) only requests subject to a budget get 503 budget_unverified, and the trigger is now usage delivery failing.
  • provider_key budgets change meaning on 1.8.0 gateways. A hard-stop provider_key budget now removes only the upstreams that use that key, instead of refusing every request on keys the provider key serves.
  • Mixed-version environments. Until a gateway is upgraded to 1.8.0 it keeps using the previous per-request budget check, including refusing whole keys for provider_key budgets and the outage behaviour above. In environments with hard-stop budgets or strict mode, a 1.6.x or 1.7.x gateway counts the new budget documents in aisix_config_unknown_kind_resources; its reload status is unaffected.

What's Changed

  • feat(mcp): x-aisix-api-key gateway credential header, oauth_delegate and Entra OBO token exchange by @jarvis9443 in #1361
  • fix(mcp): bound OBO token exchange by the server timeout; record a failed re-exchange as dispatched by @jarvis9443 in #1362
  • fix(obs): prune automatic heap dumps before counting the dump by @jarvis9443 in #1363
  • feat(proxy): make chat debug response headers opt-in via proxy.expose_debug_headers by @jarvis9443 in #1364
  • fix(obs)!: fill access-log token counts on every endpoint; adopt usage-event field names by @jarvis9443 in #1365
  • feat(budget)!: decide budgets from pushed documents; strict mode; etcd watch keepalive by @jarvis9443 in #1366
  • feat(metrics)!: every proxy route emits every per-request metric family by @jarvis9443 in #1367
  • feat(config)!: ignore unknown startup-config keys with a warning, drop retired tombstones by @jarvis9443 in #1368
  • feat(proxy): honour PROXY protocol v1/v2 headers from trusted peers by @jarvis9443 in #1374
  • build(deps): bump @hono/node-server from 2.1.1 to 2.1.5 in /tools/mcp-conformance by @dependabot[bot] in #1371
  • build(deps): bump qs from 6.16.0 to 6.17.1 in /tools/mcp-conformance by @dependabot[bot] in #1372
  • build(deps): bump undici from 7.30.0 to 7.30.1 in /tools/mcp-conformance by @dependabot[bot] in #1373
  • build(deps): bump rustls, undici and conformance-harness deps to patched versions by @jarvis9443 in #1376
  • perf(core): bump jsonschema to 0.60 so the resource validators stop holding ~115 MiB by @xz-dev in #1370
  • fix(auth): one inbound credential rule: empty headers are offered, no fall-through, case-insensitive Bearer by @jarvis9443 in #1383

New Contributors

Full Changelog: v1.7.1...v1.8.0

Contributors

dependabot, xz-dev, and jarvis9443
Assets 2
Loading