Skip to content

v9.0.0

Choose a tag to compare

@Elijas Elijas released this 06 Jul 21:08
· 157 commits to main since this release
  • Breaking: adds the public BackendLockContentionError exception and stops
    leaking raw redis.exceptions.LockError. Redis per-bucket lock contention now
    surfaces as this library exception: await_for_capacity / wait_for_capacity
    with no caller timeout retry through contention instead of raising (logging a
    throttled warning), and consume_capacity, refund_capacity,
    set_max_capacity, and reconfiguration raise BackendLockContentionError
    (chained from the underlying redis error) on lock starvation or mid-operation
    lock loss. Handlers that caught redis.exceptions.LockError must catch
    BackendLockContentionError instead; see MIGRATION.md and
    the per-bucket locking section in docs/operations.md.
  • Breaking: RedisBackendBuilder.build() / SyncRedisBackendBuilder.build()
    now raise ValueError at build time when any configured quota's
    per_seconds window is longer than bucket_ttl_seconds. That combination
    previously built without error but silently reset a drained long-window
    quota back to full capacity once an idle gap outlived the TTL. Widen
    bucket_ttl_seconds, or shorten the offending quota's per_seconds, for any
    configuration the check now rejects; see MIGRATION.md and
    the key-TTL guidance in docs/operations.md.
  • Breaking: OpenAIUsageCounter / get_encoding no longer guess a
    tokenizer from a hardcoded model-family fallback table for models the
    installed tiktoken cannot resolve on its own (for example a very new model
    release). They now raise a ValueError with upgrade/workaround guidance
    instead of either a possibly-wrong guessed encoding or a raw KeyError
    escaping from tiktoken. Code that specifically caught KeyError around
    token counting must catch ValueError instead; upgrade tiktoken or pass an
    explicit get_encoding_func to OpenAIUsageCounter for models it does not
    yet recognize. See MIGRATION.md.
  • Breaking: UsageQuotas no longer accepts the private
    _allow_empty_quotas constructor keyword; passing it now raises TypeError
    (unknown keyword argument) instead of silently building an empty quota set.
    UsageQuotas([]) still raises the same ValueError pointing you to
    UsageQuotas.unlimited(), which remains the supported way to build an
    explicit no-limit quota set. See MIGRATION.md.
  • Fixes Redis await_for_capacity / wait_for_capacity with a caller
    timeout: lock contention now retries acquisition until the caller's
    deadline instead of raising TimeoutError after
    lock_blocking_timeout_seconds (default 5s). timeout=0 still fails fast,
    and the timeout message now names lock contention as the cause instead of
    misleading capacity fields. See the per-bucket locking section in
    docs/operations.md.
  • Fixes async callback_timeout so it returns at the deadline even when a
    callback swallows cancellation, including when it is torn down via
    GeneratorExit (for example an async generator that uses the limiter being
    closed early). Previously such a callback could block acquire_capacity /
    refund_capacity for its full runtime and, on a swallowed cancellation,
    without ever logging the documented "callback exceeded timeout" warning; the
    async path now abandons the callback the same way the synchronous path
    already does, logging any error the callback raises afterward. See
    docs/observability.md.
  • Fixes a SyncRateLimiter deadlock when a PerModelConfigGetter calls back
    into the limiter (for example clear_unused_model_families) while shared
    model-family validation is in progress; the internal validation lock is now
    reentrant. acquire_capacity_for_request also now emits the same
    RuntimeWarning as acquire_capacity when called from inside a running
    event loop.
  • Fixes a spurious shutdown warning: closing a limiter with zero in-flight
    reservations no longer logs a "reservations still outstanding" warning.
  • Fixes the Redis backend hard-failing every rate-limit operation whenever the
    host's local clock lags behind the Redis server clock (for example an NTP
    outage, a paused/resumed VM, or container clock drift). Refill math already
    uses Redis server time exclusively, so a lagging local clock is harmless to
    correctness; the library now detects a genuine server-side clock jump by
    comparing consecutive Redis TIME readings against locally-elapsed
    monotonic time instead of the local wall clock, and raises only on a real
    forward jump between readings (the realistic trigger is a Sentinel/managed
    failover to a clock-skewed primary). A large divergence between the Redis
    server clock and the local wall clock now logs a one-time warning about
    possible NTP trouble instead of raising.
  • Fixes two error messages: the ValueError raised when usage exceeds a
    bucket's max capacity during acquire now names the failing quota window
    (for example "for the 60s window"), disambiguating cases where two windows
    on the same metric share a limit value; and set_max_capacity's validation
    now reports a dedicated "must be an int or float" message for wrong-typed
    inputs instead of misleadingly reusing the finite/positive-value message.
  • Fixes cancellation-path capacity refunds that fail: they now log a warning
    identifying the affected reservation instead of failing silently; the
    original cancellation error still propagates and the reserved capacity
    still recovers through normal refill.
  • Adds RateLimiter.reserve() / SyncRateLimiter.reserve(): a context
    manager over the acquire -> call -> refund cycle. It yields a handle with
    .reservation and .set_actual_usage(), refunds the unused remainder on
    normal exit (warning and conservatively refunding the full reserved usage if
    set_actual_usage was never called), and on an exception refunds with an
    optional usage_on_error (or conservatively) before re-raising the original
    exception. If a non-critical usage_on_error refund itself fails (for
    example its metric keys do not match the reservation), the reservation
    still falls back to the conservative refund instead of leaking as
    in-flight; the failure is logged, and the caller's original exception
    still propagates. See the README's "Reserve capacity around a call"
    example.
  • Fixes OpenAIUsageCounter undercounting Responses API requests that use
    text={"format": {...}} for structured output: that config is now counted
    by JSON-serializing it like response_format/tools/functions, instead of
    being walked as plain text fragments that dropped the JSON structural
    tokens (previously undercounting affected requests by roughly 62%).
  • Adds a weekly tokenizer-drift CI canary (no API key required) that checks
    the OpenAI token counter against the latest unpinned openai/tiktoken
    releases for newly-unresolvable models or untriaged request parameters.
  • Fixes the Redis ACL command list in MIGRATION.md and
    docs/operations.md: it was missing PEXPIRE (used
    by redis-py's lock extend/reacquire script) and MULTI / EXEC /
    DISCARD (used by redis-py's transaction pipelines), so a user provisioned
    strictly per the old list could pass an initial smoke test but fail under
    ordinary multi-quota usage.
  • Expands documentation coverage: the Redis ACL command list in
    MIGRATION.md now includes PTTL; its validation-error
    guidance more precisely distinguishes pydantic ValidationError from
    CardinalityLimitExceededError; the README's OpenAI example sets an
    explicit output-token budget and notes the zero-token refund on error as an
    approximation; docs/configuration.md gains a
    "Choosing reservation sizes" subsection; and
    docs/operations.md gains an "Application-facing
    errors" reference section covering DuplicateRefundError,
    UnknownReservationError, AcquireRefundFailedError, and
    CardinalityLimitExceededError.
  • Adds a test-suite safety gate that refuses to run when --redis-url points at
    a non-empty Redis database. The suite flushes that database around every test,
    so it now aborts with an actionable message instead of silently wiping data;
    set TOKEN_THROTTLE_TESTS_ALLOW_FLUSH=1 to opt in to running against a
    non-empty database.
  • Adds a test-suite thread-leak detector that fails the session if a test leaves
    a non-daemon thread or a thread-pool worker alive after a short grace period,
    catching cross-test interference that previously surfaced only as full-suite
    flakiness. Set TOKEN_THROTTLE_THREAD_LEAK_MODE=report to investigate a leak
    without failing the run.
  • Adds a stdlib-only acquire-path benchmark harness under benchmarks/
    (uv run python -m benchmarks.run, or task bench) that reports p50/p90/p99
    and ops/sec for the memory and Redis backends across sync/async and
    uncontended/contended workloads, with optional JSON output. It is not part of
    the test suite and adds no runtime dependency; absolute numbers are
    machine- and Redis-locality-dependent and meant to be read relatively. See
    benchmarks/README.md.
  • Adds a weekly scheduled soak/stress workflow (.github/workflows/soak.yml,
    also runnable on demand) that repeats the concurrency stress suites many times
    back to back, runs the property-based accounting suite, and runs a
    tightened-timing conformance pass. It exists to catch load- and soak-class
    regressions (contention and accounting bugs that only appear under sustained,
    repeated load) that the single-pass PR CI does not exercise. It changes no
    library behavior.
  • Widens the recommended pip install version bounds in the README from a
    next-minor cap to a next-major cap (for example >=8.0.8,<9.0.0 instead of
    >=8.0.8,<8.1.0), so installs can pick up minor and patch releases within the
    same major version without re-pinning. This reflects the project's semantic
    versioning guarantee that no breaking changes ship within a major.
  • Removes a stray empty __init__.py from the repository root that was never
    part of the published token_throttle package; it changes no library
    behavior.

Full upgrade guidance for the breaking changes: MIGRATION.md