Feature Request: Per-user aggregate rate limit for API access

[devkit415]

Description

API rate limits exist, but they are counted per token, not per account. A user with five tokens gets five independent budgets, and nothing in the application caps how many tokens an account may hold, so the effective ceiling for one user is "as many requests as they care to mint tokens for".

On a shared instance with dozens of programmatic clients this is the difference between a limit and a suggestion, and the currency being spent is not just requests: uploads occupy gunicorn threads, fill disk, and queue transcription jobs that cost real money at the ASR provider.

Measured on a gunicorn deployment that showed three independent counters, a single token allowed 353 successful requests in one minute against a documented limit of 120, and a token holder can multiply that figure again by creating more tokens. The request is for a second, account-wide budget that sits on top of the existing per-token budget, so that adding a token cannot add throughput.

Affected Version

v0.10.15-alpha

Requested Feature

Add a per-user rate limit that every API request authenticated by a token of that user consumes, regardless of which token carried it, and keep the current per-token limit unchanged. The two budgets should be configurable separately, with per-user defaults high enough that a legitimate client using one token sees no behavior change (for example twice the per-token default), and with a clear 429 the moment either budget is exhausted.

Alongside it, add a configurable cap on active tokens per account, because an account-wide budget makes the token cap a secondary defense rather than the only one.

Optionally, make the limiter storage configurable through RATELIMIT_STORAGE_URI so the aggregate is exact across worker processes instead of being counted once per process.

Proposed Implementation

• Hook into the one place every token route already passes through: require_scope in src/utils/token_auth.py:191-220. All 60 API v1 routes (src/api/api_v1.py:51) and all 10 webhook routes (src/api/webhooks.py:47) are decorated with it, and tests/test_api_v1_scope_coverage.py fails if a new route forgets it, so a second limiter added there covers the whole surface without touching a single route.
• Add a _USER_RATE_LIMITS table next to _RATE_LIMITS (src/utils/token_auth.py:162-166), keyed by the same read/write/process categories, backed by API_USER_RATE_LIMIT_READ, API_USER_RATE_LIMIT_WRITE and API_USER_RATE_LIMIT_PROCESS with defaults of 240 per minute, 60 per minute and 20 per minute.
• Add user_rate_key() next to token_rate_key() (src/utils/token_auth.py:177-183), returning user:<api_token.user_id>. The id is already on the APIToken row loaded for the request, so this costs no extra query. Session traffic keeps its existing per-address limits and is not affected.
• Stack the two limits inside the lazy decorator in require_scope, giving each its own scope string so the counters do not merge:

category = _rate_category(set(scopes))
state['limited'] = limiter.limit(
    _USER_RATE_LIMITS[category], key_func=user_rate_key,
    exempt_when=lambda: not is_token_request(),
    scope=lambda endpoint: f"user-{category}",
)(limiter.limit(
    _RATE_LIMITS[category], key_func=token_rate_key,
    exempt_when=lambda: not is_token_request(),
    scope=lambda endpoint: f"token-{category}",
)(f))

• Cap active tokens in create_token (src/api/tokens.py:82-140): count rows for the user where revoked is false and expires_at is null or in the future (src/models/api_token.py:48-55), and refuse with 409 and code: token_limit_reached above API_MAX_TOKENS_PER_USER (default 10, 0 meaning unlimited). Revoking a token frees a slot immediately, so the message can say so.
• Make the storage configurable: pass storage_uri=os.environ.get('RATELIMIT_STORAGE_URI') when constructing the limiter (src/app.py:175-181). Without a shared backend, Flask-Limiter keeps a counter per process (src/app.py:553), so an account-wide limit on the stock 3-worker image behaves as three times its configured value. The memory:// caveat is explicit in the Flask-Limiter documentation, which notes that each process gets its own storage.
• Document both limits in the table at docs/user-guide/api-reference.md:2027-2037 and the new environment variables in docs/user-guide/api-tokens.md, including the note that a shared storage backend is what makes the account-wide figure exact.
• Tests: extend the rate-limit section of tests/test_api_token_scopes.py with a case that creates two tokens for one user and asserts the second 429 arrives at the user budget, not the token budget, and a case that the token cap returns 409 and that revocation frees a slot.

Expected Behavior

• A user with one token and one client sees no change: the per-token budget still applies at 120 read requests per minute, and the account budget of 240 is never the binding constraint.
• A user with four tokens gets 240 read requests per minute in total, not 480. The fifth token does not add headroom, and the 429 names the account budget that was exhausted.
• The per-token budget keeps doing its job for a single runaway integration: one token cannot use the whole account budget if other integrations are active, which is why both limits are kept.
• Creating a token beyond API_MAX_TOKENS_PER_USER returns 409 with code: token_limit_reached and a message that names the current cap and points at revoking an unused token.
• With RATELIMIT_STORAGE_URI set to a shared backend, 400 requests in a minute from one account yield 240 successes and 160 rejections; with the default in-memory storage the same test yields roughly three times that, and the documentation says so.
• Session-based web traffic keeps its current per-address limits of 1000 per hour and 5000 per day (src/app.py:178), unchanged.

Root Cause Analysis

The limiter is keyed on the token row, not the account: token_rate_key returns f"token:{api_token.id}" (src/utils/token_auth.py:177-183), and require_scope applies exactly one limit built from that key (src/utils/token_auth.py:191-220). Token creation has a rate limit of 10 per hour (src/api/tokens.py:84) but no cap on the number of active tokens, and create_token never counts existing rows (src/api/tokens.py:82-140).

The result is an account-wide throughput of 120 per minute per token, which is unbounded in the number of tokens. A secondary multiplier comes from the storage backend: limiter.init_app uses the default in-memory storage (src/app.py:553), the image starts three workers (Dockerfile:148), and each holds its own counters.

A fresh minute of 400 sequential requests against one token returned 353 successes and 47 rejections, and X-RateLimit-Remaining cycled between three separate values in the same window, which is the signature of three per-process buckets rather than one shared budget.

The current behavior deviates from the intended design in two ways: the documented per-token limit understates what the deployment actually allows, and no code path aggregates traffic for an account, so the application has no way to answer "how much is this user doing right now".

Files and code paths involved: src/utils/token_auth.py (limits, keys, require_scope), src/app.py (limiter construction and defaults), src/api/tokens.py (token creation), Dockerfile (worker count), docs/user-guide/api-reference.md (published limits).

Additional Context

The ASR Voice Recorder route carries its token inside the multipart body (src/api/api_v1.py:3727-3729), so is_token_request() is false there and the per-user layer, like the current per-token layer, will not see those requests; that route keeps its own size-weighted per-address limit of 520 units per minute. This is a pre-existing boundary, not something this request needs to solve, but it should be documented next to the new limits so the behavior is not surprising.

Response headers need a note in the documentation. Flask-Limiter emits X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset for one of the applicable limits when a request has not breached any of them, and both budgets here are per-minute, so the header will report one of the two figures. The 429 body names the exhausted limit, and the tests should assert on that body rather than on the header values.

Backward compatibility is the reason for the doubled defaults: any existing single-token client stays under 240/60/20 without changing configuration, while an account that deliberately splits work across many tokens is the only one that notices. The two limits compose with the existing scopes, so a scoped token already restricted to read consumes only the read budget.

The memory:// default also explains why this feature and a shared storage backend are worth shipping together: an account-wide limit computed per process is three times its configured value on the stock image, which would make the new knob misleading in exactly the deployment where it matters most.

[murtaza-nasir]

Thank you for the detailed proposal. Requests authenticated with an API token now also count against an account budget. It applies per endpoint, with a default of twice the per-token limit, and is set with API_USER_RATE_LIMIT_READ, API_USER_RATE_LIMIT_WRITE and API_USER_RATE_LIMIT_PROCESS.

An account can hold at most 10 active tokens by default (API_MAX_TOKENS_PER_USER, 0 for no cap). Creating another one returns HTTP 409 with code: token_limit_reached. A rate-limited token request is answered with JSON that includes code: token_rate_limit_exceeded or account_rate_limit_exceeded.

Shared storage for the counters is set with RATELIMIT_STORAGE_URI, and the Redis client is now included in the image. With the default in-memory storage, the counts are kept per Gunicorn worker, as the documentation now states. This is included in v0.10.16-alpha.

[devkit415]

I hope these changes help make the application safer and more reliable. Thank you for all your hard work!

[murtaza-nasir]

Thank you for helping improve Speakr with your detailed reports and proposals.