docs: cover RateLimitPolicy, ttft_ms, runtime cooldown, on_all_filtered

[moonming]

Summary

The customer docs hadn't picked up several feature PRs that landed since the docs rebuild in #249. This PR closes those gaps:

• feat: add RateLimitPolicy entity and policy-based rate limiting #280 RateLimitPolicy — new entity with scope (api_key / model / team / member), window (second / minute / hour), and max_requests / max_tokens. Loaded from etcd directly (no admin CRUD).
• feat(proxy, admin): runtime model status, cooldown, background health check, /admin/v1/models/status #268 runtime model status — direct-model cooldown config block and routing.on_all_filtered policy were undocumented; only background_model_check had partial coverage.
• feat(obs): add ttft_ms (time to first token) to UsageEvent #276 ttft_ms — new field on UsageEvent, streaming-only.

#278 (etcd endpoint derivation on subsequent boot) is a pure internal fix with no operator-visible behavior change, so it's not included here.

Changes

File	Change
docs/configuration/rate-limits.md	rewrite around three enforced layers; add policy fields, window-mapping table, two example policies, etcd-only provisioning path
docs/configuration/api-keys.md	add team_id / owner_id bucket fields; drop stale "ApiKey-only enforcement" framing
docs/configuration/models.md	document the cooldown config block (defaults: 30s / 600s cap / [401, 408, 429, 500, 502, 503, 504] triggers / honor Retry-After)
docs/configuration/routing-and-failover.md	rewrite filter logic as a 4-step loop; document on_all_filtered (fail vs original_order)
docs/integration/errors-and-retries.md	add 503 all_candidates_unavailable row, distinct from provider_unavailable
docs/reference/headers-and-error-codes.md	add all_candidates_unavailable to the error-type list and 503 status note
docs/cloud/usage-events-and-billing.md	new "Latency Signals On The Usage Event" section covering latency_ms and ttft_ms
docs/operations/metrics-and-logs.md	new "Streaming TTFT" subsection
docs/reference/resource-schemas.md	add RateLimitPolicy to the resource map and field-shape note
docs/reference/admin-api-reference.md	new "Resources Without Admin CRUD" note for RateLimitPolicy
docs/reference/glossary.md	add RateLimitPolicy term
docs/overview/core-concepts.md	add team_id / owner_id to ApiKey; new "Rate Limit Policy" concept section
docs/overview/feature-matrix.md	tighten rate-limit row to mention all three layers

Test plan

• preview the rendered Docusaurus output and verify the new anchors resolve:
  • models.md#cooldown
  • routing-and-failover.md#all-targets-filtered-policy
  • routing-and-failover.md#runtime-filtering
  • rate-limits.md#rate-limit-policy-entities
  • rate-limits.md#troubleshooting
  • core-concepts.md#rate-limit-policy
  • admin-api-reference.md#runtime-model-status
• sanity-check that every documented field name, default, and enum value matches the schema in crates/aisix-core/src/models/schema.rs and the structs in crates/aisix-core/src/models/{rate_limit_policy,model,routing}.rs
• verify the 503 all_candidates_unavailable text matches what crates/aisix-proxy/src/error.rs emits

Summary by CodeRabbit

• Documentation
  • Expanded rate-limiting docs: multi-layer enforcement across API keys, models, and standalone rate-limit policies, plus policy schema and troubleshooting
  • Added latency signals and a time-to-first-token (TTFT) metric for streaming responses, with guidance on interpretation
  • Clarified routing, failover, cooldown, and runtime filtering behaviors and related error codes
  • Improved API key and model configuration semantics and operator guidance

[coderabbitai]

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: f654441b-c76e-4f56-b1cc-a1c0fc48a8ed

📥 Commits

Reviewing files that changed from the base of the PR and between b93cebb and 28a2dae.

📒 Files selected for processing (4)

• docs/cloud/usage-events-and-billing.md
• docs/configuration/models.md
• docs/configuration/rate-limits.md
• docs/configuration/routing-and-failover.md

✅ Files skipped from review due to trivial changes (4)

• docs/cloud/usage-events-and-billing.md
• docs/configuration/routing-and-failover.md
• docs/configuration/models.md
• docs/configuration/rate-limits.md

---

📝 Walkthrough

Walkthrough

Documentation-only updates: introduce RateLimitPolicy as an etcd-stored rate-limit resource; clarify three-layer AND-combined rate-limit enforcement (ApiKey, Model, RateLimitPolicy); expand model cooldown/probe semantics and runtime routing/filter behavior; add latency signals latency_ms and streaming ttft_ms; add all_candidates_unavailable error type and routing on_all_filtered behavior.

Changes

Rate limiting, routing, model behavior, and observability

Layer / File(s)	Summary
Multi-layer rate-limit overview and operator guidance docs/configuration/rate-limits.md, docs/overview/core-concepts.md, docs/overview/feature-matrix.md	Defines three AND-combined enforcement layers: ApiKey.rate_limit, Model.rate_limit, and scope-matched RateLimitPolicy rows; expands explanation of how inline fields map to rpm/tpm/rpd/tpd/concurrency and updates operator/troubleshooting guidance.
ApiKey semantics and matching fields docs/configuration/api-keys.md	Adds team_id and owner_id to ApiKey fields, clarifies four control layers (identity key_hash, authorization allowed_models, inline policy rate_limit, bucket identity team_id/owner_id), and documents how RateLimitPolicy rows match via team_id/owner_id.
RateLimitPolicy resource: schema, provisioning, and examples docs/reference/resource-schemas.md, docs/reference/glossary.md, docs/reference/admin-api-reference.md	Introduces RateLimitPolicy as an etcd-backed dynamic resource with required fields (name, scope, scope_ref, window), constraint that at least one of max_requests/max_tokens is set, validation/load behavior, examples, and note that admin CRUD is not exposed (direct etcd under rate_limit_policies/<id>).
Runtime routing/filtering and all-candidates-unavailable behavior docs/configuration/routing-and-failover.md, docs/integration/errors-and-retries.md, docs/reference/headers-and-error-codes.md	Replaces runtime filtering description with ordered dispatch by target state (healthy → cooldown → unhealthy); adds routing.on_all_filtered options (fail default returns 503 all_candidates_unavailable with Retry-After: 30, or original_order to ignore runtime state). Adds all_candidates_unavailable proxy error type and distinguishes it from provider_unavailable.
Direct-model probes, cooldown, and routing interaction docs/configuration/models.md	Adds cooldown to direct-model fields and expands background_model_check/probe semantics: ignore_statuses, stale_after_seconds, minimum intervals/timeouts/tokens, unhealthy-state transitions/resets, cooldown trigger conditions (statuses/timeouts), defaults, and how routing filters exclude unhealthy before cooldown candidates.
Latency observability: latency_ms and streaming ttft_ms docs/cloud/usage-events-and-billing.md, docs/operations/metrics-and-logs.md	Documents two emitted usage/metric fields: latency_ms (total elapsed request time on all request paths) and ttft_ms (streaming time-to-first-token measured at first upstream chunk with generated content). Notes that ttft_ms is omitted on non-streaming, cache-hit, and error paths and guidance for interpreting warm-up vs streaming latency.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

---

Note

🎁 Summarized by CodeRabbit Free

Your organization has reached its limit of developer seats under the Pro Plan. For new users, CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please add seats to your subscription by visiting https://app.coderabbit.ai/login.If you believe this is a mistake and have available seats, please assign one to the pull request author through the subscription management page using the link above.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.