v0.5.0 — Video generation

🚀 Get started

New to AISIX? Get the gateway running and route your first LLM request in minutes:

• 📖 Documentation — https://docs.api7.ai/ai-gateway/
• ⚡ Self-hosted quickstart (gateway + etcd via Docker Compose) — https://docs.api7.ai/ai-gateway/getting-started/self-hosted-quickstart

📦 Download

Pull the container image from the GitHub Container Registry:

docker pull ghcr.io/api7/aisix:0.5.0

Rolling tags (0.5, latest) and the docker.io/api7/aisix mirror for private/offline deployments are listed on the package page.

---

AISIX generates video. This release adds a typed video-generation surface, so prompt-to-video traffic runs through the same routing, credentials, rate limits, and guardrails as chat instead of having to go out through a raw provider tunnel. Alongside it: a new Alibaba Cloud guardrail kind, automatic Anthropic prompt caching, and a round of fixes closing paths where retries, rate limits, and usage accounting did not apply.

• /v1/videos — submit, poll, and download video tasks behind one contract, with Alibaba Model Studio, Zhipu CogVideoX, and Volcengine Ark Seedance mapped (#811, #814)
• Alibaba Cloud AI Guardrails — a guardrail kind that acts on suggestion verdicts and writes masked content back into the request instead of rejecting it outright (#799)
• Automatic Anthropic prompt caching — the gateway inserts cache breakpoints for models that opt in, so callers get the savings without touching their client code (#810)
• Local token estimation — usage is estimated instead of recorded as zero when an upstream returns no usage block (#794)
• Streaming applies routing.retries — the streaming chat path ran a separate loop that never retried (#807)
• Per-target traffic controls in group dispatch — each target's own model rate limit and client-IP allowlist now apply (#783, #786)

🎬 Video generation

• Submit → poll → download — a typed video surface following the OpenAI three-stage shape. The returned video ID carries its own routing information, so the gateway stores no task state and any instance can serve a later poll or download (#811)
• Provider mappings — Alibaba Model Studio (#811), Zhipu CogVideoX and Volcengine Ark Seedance (#814). A model whose provider has no mapping returns a typed 501
• Video requests run the full per-model path: alias resolution, caller key access checks, client-IP allowlists, model rate limits, and input guardrail scanning of the prompt. Polls are deliberately exempt from the model rate limit so a client cannot starve itself watching its own task
• Each submission records a zero-token usage event. Duration-based cost is not modelled yet, so video traffic does not draw down budgets in this release

🛡️ Guardrails

• Alibaba Cloud AI Guardrails kind — MultiModalGuard, with suggestion verdicts and mask write-back (#799)
• aisix_guardrail_latency_seconds — a per-execution latency histogram that separates guardrail overhead from upstream model latency (#798)
• Aliyun guardrail responses keep their upstream RequestId, correlated to the gateway's own request ID for cross-system troubleshooting (#781)

🧭 Routing and traffic controls

• routing.retries applies on the streaming chat path (#807)
• Group dispatch enforces each routing target's own model rate limit (#783) and client-IP allowlist (#786)
• The raw passthrough tunnel enforces the rate limit of the model named in the request body (#805)
• Azure stream_timeout bounds the gap between chunks instead of capping the whole response, so long but healthy streams are no longer cut off (#809)
• Upstream failures name the real transport fault, and the connection layer sets explicit bounds rather than inheriting library defaults (#808)

These three enforcement fixes (#783, #786, #805) close paths that previously skipped a limit. Traffic that used to slip past a per-target rate limit, a per-target IP allowlist, or the passthrough model rate limit is now limited or denied — worth a look if a deployment runs close to its configured limits.

🔭 Observability and usage

• Token counts are estimated locally when the upstream reports no usage, and the record is flagged so estimated and reported usage stay distinguishable (#794)
• Ensemble panel and judge sub-calls estimate their own usage on backends that report none (#806)
• Per-attempt error messages are no longer clipped at 256 characters (#782)
• aisix_llm_tokens_by_client_total gained a model label and is recorded on /v1/responses (#784)
• Wider built-in client_type detection, plus operator-defined user-agent mapping rules for clients the gateway does not recognize (#785)
• The status listener serves per-model runtime health, so upstream reachability can be checked without sending a model request (#778)
• The heartbeat reports the hash of the configuration the gateway has actually applied (#777)

🗄️ Prompt caching

• Models can opt into automatic Anthropic cache-breakpoint injection (#810)
• Client-supplied cache_control survives the OpenAI-to-Anthropic bridge (#804)

⚙️ Operations

• aisix export — write a resources.yaml from a running etcd store, which turns an existing deployment into a starting point for standalone mode (#761)
• admin.enabled — run without the admin listener, for deployments configured declaratively that want no write API exposed. Defaults to enabled (#791)
• Admin API write endpoints are deprecated in favor of declarative configuration. They remain fully functional with no request, response, or status-code changes; responses now carry RFC 9745 Deprecation and Link headers, and the OpenAPI write operations are marked deprecated (#776)
• Minted-token caches are keyed on the whole credential, so rotating a provider key takes effect immediately instead of after the cached token expires (#780)

What's Changed

• feat(cli): aisix export — emit resources.yaml from a running etcd store by @moonming in #761
• feat(admin): deprecate the Admin API write path in favor of declarative configuration by @moonming in #776
• feat(heartbeat): report applied config_hash for cross-plane config verification by @moonming in #777
• feat(observability): serve per-model runtime health on the status listener by @moonming in #778
• fix(providers): key minted-token caches on the whole credential by @jarvis9443 in #780
• feat(guardrails): preserve Aliyun's upstream RequestId and correlate it to the gateway request by @jarvis9443 in #781
• fix(telemetry): stop clipping the per-attempt error message at 256 chars by @jarvis9443 in #782
• feat(obs): add model label to aisix_llm_tokens_by_client_total and record it on /v1/responses by @jarvis9443 in #784
• fix(proxy): enforce each routing target's own model rate limit by @jarvis9443 in #783
• fix(routing): apply each target's client-IP allowlist in group dispatch by @jarvis9443 in #786
• feat(metrics): extend client_type coverage and add operator UA mapping rules by @jarvis9443 in #785
• docs(agents): mark member-scoped guardrails as an open gap, not a design by @jarvis9443 in #787
• feat(admin): add admin.enabled switch to run without the admin listener by @moonming in #791
• test(e2e): migrate the seed-tail tests off the Admin API onto etcd/status by @moonming in #792
• test(e2e): fix propagation race in background-health short-stale-window case by @jarvis9443 in #795
• feat(usage): estimate tokens locally when the upstream reports no usage by @jarvis9443 in #794
• test(e2e): fix propagation race in status-config resource-count asserts by @jarvis9443 in #797
• feat(guardrails): per-execution latency histogram aisix_guardrail_latency_seconds by @jarvis9443 in #798
• test(e2e): default the harness to admin-off; held-back tests opt in by @moonming in #800
• ci(release): lead draft release notes with a Get started + Download header by @moonming in #801
• test(e2e): pin the proxy health endpoints under admin-off by @moonming in #802
• fix(passthrough): enforce the body model's rate limit on the raw tunnel by @moonming in #805
• fix(proxy): estimate ensemble panel + judge sub-call usage on missing-usage backends by @jarvis9443 in #806
• fix(anthropic): preserve client cache_control through the OpenAI-to-Anthropic bridge by @moonming in #804
• feat(anthropic): auto-inject prompt-cache breakpoints when enabled on a model by @moonming in #810
• feat(videos): unified /v1/videos submit/poll/content surface (Phase 1, DashScope) by @moonming in #811
• feat(guardrails): Aliyun AI Guardrails kind (MultiModalGuard) with suggestion verdicts and mask write-back by @jarvis9443 in #799
• test(e2e): keep rate-limit bursts inside one wall-clock window by @jarvis9443 in #813
• fix(routing): honour routing.retries on the streaming chat path by @jarvis9443 in #807
• fix(upstream): name the real transport fault and bound the connection layer by @jarvis9443 in #808
• fix(azure): apply stream_timeout per chunk, not as a whole-response ceiling by @jarvis9443 in #809
• feat(videos): Zhipu (CogVideoX) and Ark (Seedance) adapters on /v1/videos by @moonming in #814

Full Changelog: v0.4.0...v0.5.0