v0.7.0 — REST APIs as MCP tools

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

• 📖 Documentation — https://docs.api7.ai/ai-gateway/
• ⚡ Gateway quickstart (one container, declarative resources.yaml) — https://docs.api7.ai/ai-gateway/getting-started/gateway-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:0.7.0

Rolling tags (0.7, latest) and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

---

A REST API becomes MCP tools without anyone writing an MCP server, and no request runs without a deadline. Point a registered MCP server at an OpenAPI document and the gateway generates one tool per operation, executes the calls, and injects the credential itself — every existing MCP control (approval gating, tool ACLs, rate limits, guardrails, usage events) applies to the generated tools unchanged. Alongside it: per-(API key × MCP server) rate limits, an upstream deadline that applies even to models that configure none, and telemetry that finally separates what the upstream took from what the caller waited for.

• OpenAPI-backed MCP servers — type: openapi plus an OpenAPI 3.x document generates one MCP tool per operation and runs tools/call as an HTTP request against the REST base URL, with the gateway-held credential injected and never exposed to the agent (#840)
• Per-(API key × MCP server) rate limits — mcp_rate_limits maps a server name to that key's own ceiling for it, so one agent looping on one server no longer drains the budget every other server draws from (#838)
• A deployment-wide upstream deadline — a model that configures no timeout is no longer unbounded; upstream.timeout_ms supplies a generous backstop against an upstream that accepts a connection and then goes silent (#844)
• Upstream and downstream latency are separate fields — the caller-facing wait is now recorded where bytes reach the client, so an output guardrail's hold-back shows up instead of being hidden behind the upstream's own number (#842, #839)
• Request bodies are uncapped by default — providers accept larger requests than any fixed gateway default, so a client that works directly against the provider no longer breaks when it goes through the gateway (#853)
• An idle gateway stays in the load balancer — /readyz no longer withdraws a healthy instance five minutes after its environment goes quiet (#841)

⚠️ Breaking changes

UsageEvent latency fields renamed

The two attempt-scoped latency fields are renamed so their scope is visible in their name, and a request-scoped field joins them:

old	new	note
latency_ms	upstream_latency_ms	same measurement, attempt-scoped
ttft_ms	upstream_ttft_ms	now measured from the start of the attempt, not from request entry
—	downstream_latency_ms	new: what the caller waited for, request-scoped

On the OTLP export the span attribute aisix.ttft_ms becomes aisix.upstream_ttft_ms, and aisix.downstream_latency_ms is added. Update any dashboard, alert, or downstream consumer that reads those names. An AISIX Cloud control plane accepts both the old and the new spelling, so a managed deployment needs no change (#842, #839).

Models with no timeout are now bounded

upstream.timeout_ms is a new deployment-wide default and it is set: 6000000 ms (6000 s). Before this release a model that set neither timeout nor stream_timeout had no upstream deadline at all — an upstream that accepted the connection and then went silent held the request open indefinitely. The value is deliberately generous, so it is a backstop rather than a responsiveness target; set a per-model timeout for the latter. To restore the previous unbounded behavior deployment-wide, set upstream.timeout_ms: 0; a single model opts out with timeout: 0.

The timeout and stream_timeout fields on a model group were dead fields — members only ever used their own values. They now apply to members that set none, resolving target → group → deployment default. A group that already carried a stream_timeout therefore starts applying it, including the pre-200 first-chunk peek it arms (#844).

request_body_limit_bytes defaults to no cap

proxy.request_body_limit_bytes defaulted to 10 MiB and now defaults to 0, which disables the cap. This applies only when the key is absent from the config — a deployment whose config file carries the value explicitly (as the shipped config.example.yaml did) keeps whatever its file says. To keep the previous behavior on a config that omits it, set request_body_limit_bytes: 10485760.

0 previously meant "reject every request with a body" and now means "no cap"; there was no way to express unlimited before (#853).

🧩 MCP

• type: openapi on an mcp_server, with an OpenAPI 3.x document under spec, generates one tool per operation. Existing rows are untouched — type defaults to mcp. Names come from a sanitized operationId capped at 128 characters, falling back to method_path; path and query parameters become schema properties and a JSON request body becomes one body property (#840)
• The gateway-held credential — bearer, api_key (default header x-api-key, overridable through the new api_key_header), or oauth2 client credentials through the existing token cache — is injected on every outbound call. Redirects are not followed, so that credential can never be re-sent to a host the operator did not configure. An upstream 401 under oauth2 invalidates the cached token so the next call re-mints (#840)
• Generated tools sit behind the same bridge surface as a real MCP upstream, so approval gating, per-tool ACLs, MCP policies, per-(key × server) rate limits, guardrails and usage events all apply to them without special-casing (#840)
• Local $ref resolution is bounded and cycle-safe (depth 16, 256 expansions) so referenced schemas keep their shape; operations whose request body has no application/json variant are skipped rather than emitting a tool that cannot succeed; header and cookie parameters are not exposed to the agent; path parameter values are rejected on traversal and percent-encoded (#840)
• Admin writes validate the spec strictly — a missing or non-object spec, Swagger 2.0 (with a conversion hint), zero generatable operations, duplicate tool names after sanitization, and api_key_header coupling each fail with a message naming the problem. Snapshot loading stays permissive: a broken row degrades like an unreachable upstream instead of poisoning the whole snapshot (#840)
• Non-2xx responses and argument mistakes, such as a missing path parameter, come back as tool-level isError results the agent can react to rather than as transport failures (#840)
• mcp_rate_limits on an API key is a map from registered server name to rps / rpm / rph / rpd / concurrency. Only tools/call is metered, so a client can still initialize and tools/list with its tool-call budget spent. Token dimensions are deliberately absent — an MCP tool call commits no tokens. The bucket key carries the API key id, so two keys with the same cap on the same server never share a counter, and the key's own rate_limit still binds on top (#838)
• The upstream.* connection settings — connect timeout, TCP keepalive, pool sizing — now apply to the rmcp transport, which had been building its HTTP client without them (#846)

⏱️ Timeouts and deadlines

• upstream.timeout_ms and upstream.stream_timeout_ms are new config-file knobs with the usual env overrides (AISIX_UPSTREAM__TIMEOUT_MS, AISIX_UPSTREAM__STREAM_TIMEOUT_MS). stream_timeout_ms defaults to 0, meaning it falls back to timeout_ms — mirroring how an unset Model.stream_timeout falls back to Model.timeout (#844)
• Deadlines resolve target → group → deployment default and are applied uniformly across every dispatch family: chat, messages on both the passthrough and cross-provider paths, responses, count_tokens, completions, embeddings, rerank, images, audio, videos, jobs, passthrough, the ensemble panel and judge, and the realtime idle cap (#844)
• For streaming, the dedicated stream knob outranks the generic one at every level, and resource-level values are exhausted before the deployment default is consulted (#844)
• The deployment default deliberately does not arm the pre-200 first-chunk peek. Peeking withholds the 200 until the first token, which is the slow-first-token failover contract — doing that for every model would silence the SSE heartbeats whose whole purpose is covering a slow first token. With only the default budget, a first-chunk stall surfaces as an in-band timeout after the 200 (#844)
• Bedrock honors the model deadline and the connect timeout on the SDK client. The AWS SDK was constructing its own HTTP client with its own defaults, so a Bedrock model's configured timeout was not being enforced (#845)

🔭 Observability

• downstream_latency_ms is stamped where bytes are handed to the client, not where the upstream chunk arrives. Under a masking output guardrail the stream is held back until the whole response scans clean — measured on the same request, 6 ms at the upstream chunk versus 750 ms+ at the actual handoff. All four streaming pumps are wired: the chat renderer, the Anthropic native passthrough and its cross-provider encoder, and both responses paths (#842)
• downstream_latency_ms - upstream_ttft_ms is the wait the final attempt's upstream did not account for, which makes "gateway or upstream?" answerable from telemetry alone. On a request that retried it also contains the earlier attempts and the backoff, so read it together with attempt_index before attributing the difference (#842)
• The winning attempt's latency is scoped to that attempt. A request that failed over previously reported the winner's latency measured from request entry, so it carried every earlier attempt's time and the backoff — which made the retry look like a slow upstream (#839)
• Prometheus histograms no longer grow without bound. Series handles for the request and usage metric families are cached rather than re-resolved per request (#850, #856, #857)

🔌 API surface

• The 413 surface is uniform across every route that accepts a body. JSON handlers, raw-bytes handlers and multipart handlers all map an over-limit body to the caller's standard error envelope; /mcp and /a2a return the enveloped 413 instead of a bare 400. Previously five JSON handlers and both raw-bytes handlers leaked a stock text/plain rejection, and the multipart handlers folded a correctly classified 413 into a generic 400 (#853)
• Malformed JSON on those converted handlers now also returns the standard enveloped 400 (#853)
• The duplicate-Content-Length rejection — request-smuggling hygiene — keeps firing regardless of how the body cap is set (#853)

☸️ Deployment

• /readyz gates on what it can answer: not draining, and configuration applied at least once. The event-recency condition and its READYZ_STALE_AFTER knob are gone. That condition measured elapsed time since the last configuration event, which an unchanging environment simply does not produce — so every replica crossed the threshold at the same moment and the Service lost all of its endpoints while perfectly healthy. Configuration freshness stays fully observable through /admin/v1/health, /status/config and the aisix_config_* metrics (#841)
• Release binaries keep their symbol table, so a flame graph taken against a shipped binary is readable without a special build (#849)

🚀 Performance

• Per-request state clones are constant-cost rather than growing with the size of the resolved configuration (#851)
• Prometheus series handles for the request and usage families are resolved once and cached instead of being looked up on every request (#856, #857)

What's Changed

• feat(mcp): per-(API key × MCP server) rate limiting by @jarvis9443 in #838
• fix(telemetry): scope the winning attempt's latency_ms to that attempt by @jarvis9443 in #839
• feat(mcp): expose REST APIs as MCP tools from an OpenAPI spec (type: openapi) by @jarvis9443 in #840
• fix(health): /readyz must not withdraw an idle gateway by @jarvis9443 in #841
• feat(telemetry): split upstream and downstream latency on UsageEvent by @jarvis9443 in #842
• docs: bring the README in line with what the gateway actually ships by @moonming in #843
• feat(config): deployment-wide default for upstream request/stream timeouts by @jarvis9443 in #844
• fix(bedrock): enforce the model deadline and connect timeout on the SDK client by @jarvis9443 in #845
• fix(mcp): apply the upstream.* connection settings to the rmcp transport by @jarvis9443 in #846
• fix(metrics): prevent unbounded histogram memory growth by @moonming in #850
• build: keep the symbol table in release binaries for field profiling by @membphis in #849
• test(mcp): unique client_id per oauth test config — port reuse poisoned the global token cache by @jarvis9443 in #855
• perf(proxy): make request state clones constant-cost by @moonming in #851
• perf(metrics): cache request series handles by @moonming in #856
• feat(config): request_body_limit_bytes defaults to 0 = no cap; uniform 413 surface by @jarvis9443 in #853
• perf(metrics): cache usage series handles by @moonming in #857

New Contributors

• @membphis made their first contribution in #849

Full Changelog: v0.6.0...v0.7.0