Skip to content

feat: build a key's Limiter from its key, for datastore-backed limiters - #14

Merged
christiangda merged 1 commit into
mainfrom
feat/key-bound-limiter-factory
Aug 25, 2026
Merged

feat: build a key's Limiter from its key, for datastore-backed limiters#14
christiangda merged 1 commit into
mainfrom
feat/key-bound-limiter-factory

Conversation

@christiangda

Copy link
Copy Markdown
Contributor

The gap

A Limiter is bound to exactly one key — Allow() and Wait() take no
arguments, so the instance is the bucket.

For an in-process limiter that's invisible: every bucket is equivalent, so
newLimiter func() Limiter suffices. For a limiter whose counter lives
somewhere else — Redis, Valkey, any shared store — it's the whole problem: such
a limiter must know which remote key is its own, and nothing in the API ever
told it.

The fix

bl := ratelimiter.NewBucketLimiter(nil, time.Minute,
    ratelimiter.NewInMemoryStorage[string, ratelimiter.Limiter](),
    ratelimiter.WithLimiterFactoryForKey(func(key string) ratelimiter.Limiter {
        if shared != nil {
            return sharedLimiter{client: shared, key: "rl:" + key}
        }
        return ratelimiter.RateLimiter{Limiter: rate.NewLimiter(limit, burst)}
    }),
)

That's the entire seam for "shared budget when the datastore is there,
in-process when it isn't"
. Note the storage in both branches: the ordinary
in-memory one. It only caches handles; the shared state lives in the datastore,
inside the Limiter.

Why not a distributed Storage — the thing everyone tries first

GetOrAdd hands the caller the Limiter and the caller calls Allow() on it.
The manager never writes backStore isn't called by it at all. So a
Storage that serialised bucket state to Redis would deserialise a full
bucket
on every request, let the caller spend a token from an object nobody
persists, and drop it. The limit would never be reached, and nothing would look
wrong.

docs/CUSTOM_STORAGE.md already said so. This PR gives people the thing to
do, instead of only the thing not to do.

The previous advice — use Storage.LoadOrStore as a key→limiter resolver — still
works and is still documented, but it reinterprets an interface whose job is to
hold values, and hides construction where nobody looks for it.

Option is deliberately not made generic

Changing it to Option[K] would force every existing WithClock(now) call to be
explicitly instantiated — breaking source compatibility for every current user.
The factory is stored as any and recovered with a checked assertion at
construction
, so a key-type mismatch panics there rather than surfacing as a
nil limiter on the first request for a new key. Supplying neither factory now
panics at construction too, for the same reason.

Fully backward compatible: newLimiter keeps working and is only ignored when
the option is supplied.

Tests — each verified to fail by mutating what it guards

  • the key reaches the factory, and one key's spending doesn't affect another ✅
  • state survives eviction of the handle — the property separating a
    datastore-backed limiter from an in-process one ✅
  • the key-aware factory wins when both are supplied
  • no factory at all panics at construction ✅
  • a mismatched key type panics ✅
  • concurrent first use returns one instance ✅

Two of these were rewritten because the first version couldn't fail, which
is worth calling out:

  • the mismatch test only asserted "a panic happened" — with the type check
    removed it still passed, because the both-factories-nil guard fired instead.
    It now asserts which panic.
  • the concurrency test counted tokens against shared state, which passes whether
    or not creation is atomic, since every instance addresses the same budget. It
    now asserts instance identity.

examples/keyfactory

Shows the difference in six lines of output — evict the handle and a shared
limiter stays spent, an in-process one comes back full:

$ go run ./examples/keyfactory              $ go run ./examples/keyfactory -shared=false
alice  allow=true                           alice  allow=true
alice  allow=true                           alice  allow=true
alice  allow=false                          alice  allow=false
bob    allow=true                           bob    allow=true

after evicting alice's handle:              after evicting alice's handle:
alice  allow=false                          alice  allow=true

Docs

CUSTOM_STORAGE.md and the README updated. The distributed section also stops
implying a Lua script is the only way to be atomic: a token bucket is a
read-modify-write and needs EVAL or WATCH/MULTI/EXEC, but a
sliding-window counter is a single atomic INCR. Choosing the algorithm is a
legitimate alternative to choosing a scripting engine.

Gates

go test -race -coverprofile -covermode=atomic -tags=unit ./...100.0% of
statements
, unchanged · go vet · gofmt clean.

🤖 Generated with Claude Code

A Limiter is bound to exactly one key -- Allow and Wait take no arguments, so
the instance IS the bucket. For an in-process limiter that is invisible: every
bucket is equivalent, so `newLimiter func() Limiter` suffices. For a limiter
whose counter lives somewhere else -- Redis, Valkey, any shared store -- it is
the whole problem, because such a limiter has to know WHICH remote key is its
own, and nothing in the API ever told it.

WithLimiterFactoryForKey(func(K) Limiter) closes that gap.

    bl := ratelimiter.NewBucketLimiter(nil, time.Minute,
        ratelimiter.NewInMemoryStorage[string, ratelimiter.Limiter](),
        ratelimiter.WithLimiterFactoryForKey(func(key string) ratelimiter.Limiter {
            if shared != nil {
                return sharedLimiter{client: shared, key: "rl:" + key}
            }
            return ratelimiter.RateLimiter{Limiter: rate.NewLimiter(limit, burst)}
        }),
    )

That is the entire seam for "shared budget when the datastore is there,
in-process when it is not" -- and note the storage in both branches is the
ordinary in-memory one, because it only caches handles. The shared state lives
in the datastore, inside the Limiter.

WHY NOT A DISTRIBUTED Storage, which is what everyone tries first: GetOrAdd
hands the caller the Limiter and the caller calls Allow() on it. The manager
never writes back -- Store is not called by it at all. So a Storage that
serialised bucket state to Redis would deserialise a FULL bucket on every
request, let the caller spend a token from an object nobody persists, and drop
it. The limit would never be reached, and nothing would look wrong.
docs/CUSTOM_STORAGE.md already said so; this commit gives people the thing to
do instead of only the thing not to do.

The previous advice was to use Storage.LoadOrStore as a key->limiter resolver,
because it was the only place that saw both the key and a shared client. That
still works and is still documented, but it reinterprets an interface whose job
is to hold values and hides construction where nobody looks for it.

Option is deliberately NOT made generic. Changing it to Option[K] would force
every existing WithClock(now) call to be explicitly instantiated, breaking
source compatibility for every current user; the factory is stored as any and
recovered with a checked assertion at construction, so a key-type mismatch
panics there rather than surfacing as a nil limiter on the first request for a
new key. Supplying neither factory now panics at construction too, for the same
reason.

Tests, each verified to fail by mutating the code it guards -- and two of them
were rewritten because the first version could not fail:

  - the key reaches the factory, and one key's spending does not affect another
  - state survives eviction of the handle, which is the property that separates
    a datastore-backed limiter from an in-process one
  - the key-aware factory wins when both are supplied
  - no factory at all panics at construction
  - a mismatched key type panics, and the test asserts WHICH panic: checking
    only that "a panic happened" passed with the type check removed, because
    the both-factories-nil guard fired instead
  - concurrent first use returns ONE instance, asserted by identity: the first
    version counted tokens against shared state, which passes whether or not
    creation is atomic, since every instance addresses the same budget

examples/keyfactory shows the difference in six lines of output: evict the
handle and a shared limiter stays spent while an in-process one comes back full.

docs/CUSTOM_STORAGE.md and README updated. The distributed section also stops
implying a Lua script is the only way to be atomic -- a token bucket is a
read-modify-write and needs EVAL or WATCH/MULTI/EXEC, but a sliding-window
counter is a single atomic INCR, and choosing the algorithm is a legitimate
alternative to choosing a scripting engine.

Coverage stays at 100% of statements.
@christiangda christiangda self-assigned this Aug 25, 2026
@christiangda
christiangda merged commit 8923b54 into main Aug 25, 2026
5 checks passed
@christiangda
christiangda deleted the feat/key-bound-limiter-factory branch August 25, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant