Skip to content

v0.10.0 — A gateway for agent traffic

Choose a tag to compare

View all tags
@github-actions github-actions released this 20 Aug 11:19
· 167 commits to main since this release
v0.10.0
e8d0e04
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:0.10.0

Rolling tags (0.10, latest) and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

AISIX becomes a gateway for agent traffic, not only for LLM calls.

Highlights

  • Forward-proxy traffic gets a real resource. The implicit /passthrough/:provider tunnel is replaced by explicit PassthroughRoute resources that match on path prefix, inbound Host, or both — so an IDE or coding agent can be pointed at the gateway and audited without touching the client (#982, #984, #985, #986).
  • MCP 2026-07-28, with the handshake-free discovery lifecycle and stateless transport, plus a per-server protocol_version pin for upstreams that no longer answer the older handshake (#980, #981).
  • MCP tool access is one rule across three layers — environment policy, team policy, and the key's own block — with allow lists intersecting and deny lists combining (#992, #993).
  • Anonymous MCP access and OAuth 2.1 resource-server discovery, so credential-less clients can reach chosen entries as a bound principal, and standard clients can discover where to get a token (#859, #989).
  • Consistent hashing and priority tiers for model groups (#997).
  • A drain window on shutdown, so connections a load balancer routes inside its own detection window are still served (#995).

🧭 Routing

strategy: consistent_hash pins each session to one target by hashing a request key over a ring: the same key keeps landing on the same target, weights scale each target's share, and when a target fails only its own sessions move to their ring successors. The key comes from a configurable source chain (hash_on: header, cookie, API key, or client IP). Independently, every target takes a priority — targets partition into tiers, higher is preferred, and a lower tier receives traffic only once every target above it has failed or been health-filtered, so a backup pool can sit idle behind an active one (#997).

🧩 MCP

The tool ACL collapses into three layers of identical shape — the environment policy, the caller's team policy, and the key's mcp_access block — whose allow lists intersect and deny lists combine. A layer that is absent imposes no constraint; with no layer at all the grant is empty, so access is always granted explicitly (#992, #993).

/mcp and /mcp/{server} serve the 2026-07-28 revision. A registered server takes an optional protocol_version to pin the revision used for its upstream session — selection is explicit in both directions, with no probing and no silent fallback (#980, #981). An environment can open chosen entries to credential-less clients as a nominated API-key principal, gated by a mandatory source-CIDR allowlist (#989), and can publish OAuth 2.1 protected-resource metadata with a matching WWW-Authenticate challenge (#859).

Guardrails can be scoped to a single registered MCP server, and now scan a tool's structured output as well as its text blocks (#979).

🔌 Passthrough

A passthrough route binds one gateway entry to one upstream target. Host matching runs pre-routing so forward-proxy traffic arriving with its original Host can claim a path the gateway also uses; path matching runs as the router's fallback so a route can never shadow the gateway's own API. Callers authenticate with a gateway key, with the key in a header you name, or anonymously behind a source-CIDR allowlist; the upstream credential is either injected from a provider key or forwarded from the caller verbatim. The request envelope is detected per request and drives guardrail extraction, audit capture and token accounting, so non-LLM traffic records no phantom tokens (#982, #984, #985, #986, #988).

🔭 Observability

Exported traces now form a hierarchy: an HTTP SERVER span, a logical client span covering every retry and failover, and one child per upstream attempt. Ids are minted once per request, so a delivery retry re-sends byte-identical ids. A valid inbound traceparent is honored; a malformed one falls back to a local root rather than failing the request. The caller's traceparent/tracestate are no longer forwarded to providers (#996).

The deployment and fallback counters are now emitted, pre-dispatch failures are kept out of them, and failed requests are attributed to the caller that was resolved (#972, #977, #987).

🚦 Operations

On SIGTERM the gateway reports itself not-ready immediately, then keeps accepting connections for at least shutdown.min_drain_secs (default 30s) before closing its listener, adding Connection: close to HTTP/1.1 responses so pooling clients retire connections as they use them. The window is a minimum, not a deadline (#995).

Release images are built with profile-guided optimization (#975), and the allocator returns freed memory to the OS after load subsides (#970).

🐛 Notable fixes

  • Streamed audio transcription was billed as free. The SSE decoder ended an event only on \n\n, while providers frame transcription streams with \r\n\r\n, so the terminal event carrying the token counts was never decoded. Events now end on any pair of line terminators — this affects every streaming bridge, not only audio (#1003).
  • Streamed transcription is relayed live instead of being answered from a fully read body, and /v1/audio/speech streams as it arrives (#1000).
  • Strict validation names the offending setting when it rejects configuration a model kind never reads (#971).

⚠️ Breaking changes

  • The implicit passthrough tunnel is gone. /passthrough/:provider/*rest no longer resolves by provider name; unclaimed /passthrough/* paths answer 410 Gone with error.code: endpoint_removed. Recreate the old prefix as an explicit route — path_prefix: /passthrough/<provider>, target_url set to the old API base, provider_key_id set to the key it borrowed — and client URLs keep working byte for byte. Routes are single-attempt relays (no upstream retries) and resolve no model (no cooldown cross-marking). Access requires an explicit allowed_routes grant on the caller key (#982).
  • strategy: weighted and routing.sticky are removed. Weight is now a per-target attribute under every strategy. Migrate weighted → round_robin (weights kept, now smooth weighted round robin) and weighted + sticky → consistent_hash. A stored row still naming weighted fails to load rather than changing meaning silently; sticky cohorts reshuffle once, since consistent hashing builds its ring differently (#997).
  • McpPolicy.mode, McpAccess.mode and ApiKey.allowed_tools are removed. mode: all becomes allow: ["*"], mode: none becomes allow: [], and the key's grant is mcp_access.allow. allow is required on every layer, so a layer that only subtracts must spell its allow side ["*"]. Two semantics change: a team policy now narrows the environment layer instead of replacing it, and a key with no block follows the policy layers instead of standing outside them. If you have configured MCP tool policies, re-check them after upgrading — the resulting access is recomputed under these rules (#992).
  • A guardrail blocking an MCP tool call now answers result.isError: true at HTTP 200, instead of JSON-RPC error.code: -32600. Clients matching on -32600 must read result.isError (#979).
  • An unsupported MCP-Protocol-Version is now refused inside the JSON-RPC envelope (HTTP 400, -32600, supported list in error.data) instead of a bare text/plain 400. 2024-11-05 is no longer advertised (#980).
  • Exported OTLP traces changed shape from one flat span per usage event to a SERVER → logical → attempt hierarchy. Span-count alerts will see structural spans appear; the attempt span carries the full attribute set and is identifiable by aisix.attempt_index (#996).
  • /v1/audio/speech latency is now time to first byte, since the response streams instead of buffering (#1000).

What's Changed

  • perf(server): enable jemalloc background purge thread + post-load RSS decay leg by @membphis in #970
  • fix(schema): name the dead knob when strict model validation rejects it by @jarvis9443 in #971
  • build: fail-closed PGO for release images by @membphis in #975
  • fix(obs): emit the deployment and fallback counters that had no caller by @jarvis9443 in #972
  • fix(obs): keep pre-dispatch failures out of the deployment counters by @jarvis9443 in #977
  • feat(mcp): scan structured tool output, block as a tool error, scope guardrails per server by @jarvis9443 in #979
  • feat(mcp): upgrade rmcp 1.8.0 → 3.1.2 — serve MCP 2026-07-28, pin MRTR/cache/version defaults by @moonming in #980
  • feat(mcp): per-server protocol_version for modern-only (2026-07-28) upstreams + conformance CI gate by @moonming in #981
  • feat(passthrough): explicit PassthroughRoute resources replace the implicit provider tunnel by @jarvis9443 in #982
  • fix(passthrough): OTLP route/identity attribution, route on rejected events, header_key 401 names its header by @jarvis9443 in #983
  • fix(passthrough): forward-proxy routes may claim reserved prefixes and mirror the whole path by @jarvis9443 in #984
  • feat(passthrough): openai_responses protocol hint for agent traffic by @jarvis9443 in #985
  • feat(passthrough): per-request envelope detection, drop protocol/streaming config by @jarvis9443 in #986
  • fix(obs): attribute failed requests and label the usage-event/cancel counters by @jarvis9443 in #987
  • feat(mcp): OAuth 2.1 resource-server discovery surface for /mcp by @membphis in #859
  • feat(mcp): anonymous access to named /mcp entries by @jarvis9443 in #989
  • fix(passthrough): report every usage dimension a detected envelope carries by @jarvis9443 in #988
  • docs(readme): MCP gateway protocol-generation support in the feature list by @moonming in #990
  • refactor(mcp): resolve tool access as three intersecting layers by @jarvis9443 in #992
  • fix(mcp): keep loading api_key rows projected before the layered ACL by @jarvis9443 in #993
  • feat(routing): consistent_hash strategy + priority tiers; fold weighted into round_robin by @jarvis9443 in #997
  • feat(server): keep serving through a drain window before closing the listener by @jarvis9443 in #995
  • fix(mcp): consume the control plane's legacy mcp_access mode tombstone by @jarvis9443 in #1002
  • fix(audio): relay a streamed transcription live, and quiet the metadata-probe WARNs by @jarvis9443 in #1000
  • fix(sse): frame events on any line-terminator pair, not just \n\n by @jarvis9443 in #1003
  • feat(guardrails): local CPU embedding-model guardrail MVP by @membphis in #999
  • feat(obs): OTLP trace foundation — stable IDs, W3C traceparent, span hierarchy by @moonming in #996

Full Changelog: v0.9.0...v0.10.0

Contributors

membphis, moonming, and jarvis9443
Assets 2
Loading