feat: Backend — one limit shared across processes, any KV store

[christiangda]

Builds on #14 (merged), which supplies the WithLimiterFactoryForKey seam.

The gap

Storage holds limiters in this process. There was no seam for holding the
count somewhere else — so a limit shared across instances was out of reach
without abandoning this package.

type Backend interface {
	Take(ctx context.Context, key string, limit Limit, cost int) (Decision, error)
}

Implement it over Valkey, Redis, DynamoDB, Postgres — or use the bundled
MemoryBackend — and every process shares one budget.

Why Take-shaped and not Get/Set-shaped

The obvious design is a small key-value interface with the token arithmetic in
this package. That works in one process and is a race everywhere else:

read (tokens, ts) → refill by elapsed time → compare → write back

Two processes read the same state, both decide they may proceed, both write, and
the limit silently becomes 2×. Making it safe needs a transaction with a
retry loop on a key that is contended by definition, or a server-side script —
neither expressible through Get/Set.

So the decision has to run where the state lives, and the interface has to be
the decision. That is also precisely why Storage cannot do this: it is a
container of Limiter values in this process, and its Load/Store shape is
the Get/Set shape.

You want…	Extension point
a different in-process container	Storage
a different algorithm	Limiter
a shared limit	Backend

What BackendLimiter adds

• A local fallback. Neither answer to "the datastore is down" is acceptable
  alone: refusing turns a blip into a total outage; allowing deletes the limiter
  exactly when it is needed. With a fallback there is nothing to choose between —
  an outage degrades to per-process limiting, which is what you had before adding
  a backend at all.
• A circuit breaker. Falling back is only half a fallback. Without one,
  every request during an outage pays a failed round trip before reaching the
  local answer it was always going to get.
• A degraded signal. A limiter silently enforcing N× the intended limit is
  invisible from a request. Alert on the state, not the error rate.

MemoryBackend is a window counter on purpose

Even though this package has a better in-process limiter. It is the same
algorithm a datastore backend runs, so moving between them does not move the
behaviour underneath you. RateLimiter stays the right choice when you want the
best in-process limiter — documented in TOKEN_BUCKET.md with the trade-off
table.

Two bugs the tests found — in code I had already written and believed

1. The single-prober guard did not work. It used a mutex with
   defer Unlock(), which releases when shouldAsk returns — nanoseconds
   later, long before the backend answers. Every caller acquired it in turn.
   Measured: 20 of 20 racers reached the backend, the exact thundering herd
   the guard exists to prevent. Now a CAS held across the call.
2. A failed probe never released the slot, so the breaker wedged shut for
   ever: the backend was never re-tested and the limiter stayed degraded
   permanently, including long after recovery. Worse than the outage, because
   it does not end when the outage does.

Both have tests now, both verified to fail by reverting the fix.

Benchmarks drove an API addition

BackendLimiter.Allow measured 391 ns/op against a backend answering in
100 ns — all of it context.WithTimeout arming a timer per request.

with a timeout:  391 ns/op
without:         110 ns/op

WithoutBackendTimeout removes the deadline for in-process backends, documented
as never for a network backend, where the deadline is what bounds a hung
datastore.

Tests

Every one verified to fail by mutating the code it guards — 7 mutations.
Coverage 100.0%, the level this repo was at before this work.

Docs — all of them

File
docs/BACKENDS.md	new; includes a ~25-line Valkey implementation
doc.go	rewritten — it claimed distributed limiting was "out of scope for the bundled types", which this makes false
docs/TOKEN_BUCKET.md	"deliberately out of scope" replaced with the real trade-off table
docs/CUSTOM_STORAGE.md	routes readers to BACKENDS.md instead of only explaining what not to do
docs/MIGRATION.md	notes the Backend release is additive
README.md	Backends section; the "single-process only" claim qualified

All four cross-link; link check clean. Runnable: go run ./examples/backend.

Dependencies and toolchain

golang.org/x/time is already at v0.15.0, the latest — go get -u ./... and
go mod tidy produce no change, and the dependency list stays one entry.

The go directive moves 1.26.0 → 1.27.0, which is what CI actually builds
with (go-version-file: ./go.mod) and matches the consumers. go fix then had
one thing to say — a three-line clamp collapsed to max(...), which happened to
be the last statement no test reached, taking coverage back to 100%.

Reviewers: raising the go directive raises the minimum Go version for
everyone importing this library. It is the one part of this PR that is not
purely additive. Everything else keeps every existing call working — Option
was deliberately not made generic, or every current WithClock(now) would
need explicit instantiation.

Unrelated finding

CI runs go-test-coverage --config=./.testcoverage.yml and that file does not
exist in the repo. It exits 0 because the step pipes through tee without
pipefail, so the coverage gate silently passes without ever running. Not
touched here — adding the config would start enforcing a threshold nobody has
chosen.

🤖 Generated with Claude Code