Skip to content

Spec Rate Limiting and Quotas

mavrovde edited this page Nov 1, 2026 · 1 revision

Spec: Rate Limiting, Quotas and Priority

Part of the Technical Specification. Requirement prefix: RATE. This page makes Rate Limits and Priority normative.

1. Gates

# Gate Algorithm (abstract) Key Refusal
1 Per-IP flood control Token bucket per client address (rate ratelimit.anon_rps, burst ratelimit.anon_burst) address 429 rate-limited
2 Per-key rate Token bucket, equivalently the generic cell rate algorithm (GCRA): a request is admitted if its theoretical arrival time does not exceed now plus the burst tolerance key id 429 rate-limited
3 Quota Fixed-window counters per key: one daily window (reset at 00:00 Europe/Berlin) and one monthly window (reset at 00:00 Europe/Berlin on the first) key id 429 quota-exceeded
4 Priority admission Bounded priority queues per tier in front of a permit pool equal to the storage connection pool; strict priority with per-tier reservations, an in-flight cap for bulk, aging, and refusal when the estimated wait exceeds the remaining deadline tier 429 overloaded (or 503)

2. Headers

Every keyed response, including errors after the key was identified, carries the IETF RateLimit fields (structured fields, RFC 9651):

RateLimit-Policy: "burst";q=20;w=1, "day";q=10000;w=86400, "month";q=200000;w=2592000
RateLimit: "burst";r=17;t=1, "day";r=8412;t=38211

q is the quota units of the policy, w its window in seconds, r the remaining units and t the seconds until the window resets. The values above are illustrative, not plan values. Retry-After carries whole seconds.

3. Admission

flowchart LR
  R[Request past rate and quota] --> E{Estimated wait fits<br/>the remaining deadline}
  E -->|no| X[429 overloaded with Retry-After]
  E -->|yes| Q{Tier queue has room}
  Q -->|no| X
  Q -->|yes| W[Wait in the tier queue]
  W --> P[Permit granted by strict priority,<br/>reservations, bulk cap, aging]
  P --> S[Storage read with the remaining budget]
Loading

Requirements

ID Requirement Verification
RATE-001 A per-IP token bucket MUST apply to every request before key validation, including unauthenticated and service endpoints; the client address MUST be taken from the forwarded header only when the immediate peer is in server.trusted_proxies. Load
RATE-002 Per-key rate and burst MUST come from the key's plan and MUST be enforced with a token-bucket-equivalent algorithm without per-request storage writes. Unit
RATE-003 Daily and monthly quotas MUST reset at midnight Europe/Berlin (quota.reset_tz), correctly across daylight-saving changes. Unit
RATE-004 Every request that passes authentication, including 304 and 4xx after authentication, MUST count one unit against the quota; composed features MUST count one unit, not one per part. Unit
RATE-005 OPTIONS, service endpoints and requests refused before authentication MUST NOT count against a key's quota. Handler
RATE-006 Every keyed response MUST carry RateLimit-Policy and RateLimit in the syntax of the draft revision pinned in the OpenAPI document; switching them off (ratelimit.headers) MUST be possible. Handler
RATE-007 Every 429 MUST carry Retry-After: the time to the next token for rate-limited, the time to the reset for quota-exceeded, and an estimate of the queue drain for overloaded. Handler
RATE-008 Admission permits MUST equal the storage connection pool size; nothing may wait inside the pool. Review
RATE-009 A released permit MUST go to the highest non-empty tier, respecting per-tier reservations (tiers.*.reserved) and the bulk in-flight cap (tiers.bulk.max_inflight). Unit
RATE-010 A waiter older than admission.aging_after MUST be promoted one tier, so that no tier starves. Unit
RATE-011 The controller MUST refuse at once when the estimated wait (queue position times a moving average of service time, admission.ewma_alpha) exceeds the remaining deadline minus deadline.reserve. Unit
RATE-012 Under overload, bulk MUST be shed first, then standard; critical only when its own queue is full. Load
RATE-013 A composed feature MUST take one permit per parallel part, at most compose.max_parallel. Unit
RATE-014 Usage MUST be metered per key, hour and operation (requests, refusals by reason, 304, storage time) in memory and flushed in batches every metering.flush_interval; a crash loses at most one interval, and this MUST be documented. Unit
RATE-015 Per-IP limiter state MUST be held in memory only and MUST expire after ratelimit.ip_state_ttl (design 10 min); client addresses MUST NOT be persisted by the limiter. Security
RATE-016 Fairness: within one tier, waiters MUST be served first in, first out; no single key MAY hold more than tiers.per_key_inflight permits at once. Unit
RATE-017 Acceptance (overload scenario on a test stack, never production): a steady critical stream keeps its latency target while a bulk flood far above capacity receives 429 with Retry-After, and no request runs past 2 s ([[Benchmark Plan Benchmark-Plan]] check 3).

Home

About

Design

Data Model

Technical specification

Testing

Engineering

Technology choice

Reference

Clone this wiki locally