Skip to content

tps/tph: per-second / per-hour token-rate counters need a different counter strategy than FixedWindowCounter + post-deduct add #396

Description

@moonming

Context

Follow-up to PR fixing api7/AISIX-Cloud#426. That PR adds per-second (`rps`) and per-hour (`rph`) request counters to `RateLimit` so a standalone `rate_limit_policies` row with `{window: second, max_requests: N}` actually enforces 1-second windows (was incorrectly upscaled to `rpm = N*60`, allowing 60× bursts).

Tokens at the same windows (`tps`/`tph`) are intentionally NOT shipped in that PR. This issue tracks the design work.

Why tokens differ from requests

Token counts arrive post-deduct (after the upstream LLM response completes with its terminal `usage` frame). The existing `tpm` counter uses `FixedWindowCounter::add(now, delta)` which calls `roll_if_stale` before adding. At a window boundary, this silently grants a freebie:

  • Request fires at `t=0.9s`, runs 200ms, terminal usage arrives at `t=1.1s`
  • `s.tps.add(t=1.1, 1000)` → `roll_if_stale` rolls the stale 1s window, sets `count=1000` in the new window
  • Previous window's actual 1000-token consumption is dropped on the floor
  • Pre-commit at `t=1.2` sees `count=1000` which is correct for the current window, but the previous window's overshoot was effectively free

For `tpm` (60s window) this is sub-optimal but bounded: response time / window = ~30s/60s = 50% worst-case freebie ratio.

For `tps` (1s window) this is exploitable: response time / window = ~5s/1s = 500% — a customer can routinely 5× their declared per-second token budget by timing requests to land on window boundaries.

For `tph` (3600s window) the same race applies but ratio drops to ~5s/3600s = 0.14% — likely acceptable but worth confirming.

Proposed designs (to investigate)

  1. Sliding window — carry overflow from previous window so a tail-arriving token batch is charged against the bucket it actually consumed
  2. Pre-deduct estimate + reconcile — reserve estimated max_tokens at pre-commit, refund unused after usage frame
  3. Track last-N windows + decay — generalized sliding decay
  4. Defer in product — make per-second / per-hour tokens not a supported window combination; reject at `validate_rate_limit_policy` schema layer

Need to compare against reference implementations (LiteLLM, Portkey) — see what they do for sub-minute token budgets, if anything.

Scope

This issue blocks the symmetry of:

  • `crates/aisix-proxy/src/quota.rs::policy_to_rate_limit` for the `max_tokens` arm under `window=second` and `window=hour`
  • `crates/aisix-ratelimit/src/limiter.rs` token-counter wiring at sub-minute scales

Until this lands, `policy_to_rate_limit` should silently ignore `max_tokens` on sub-minute windows (with a tracing log or schema-side rejection). The cp-api admission already accepts `max_tokens` on any window per `crates/aisix-core/src/models/schema.rs:531`; that's a contract gap surfaced by this work.

Filed by @claude (AI assistant) during the #426 fix work. cc @moonming for triage.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions