Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
881e8fe
refactor(backends): share the entry codec and key-scan helpers
allen0099 Sep 5, 2026
d5db316
fix(memcached): pool connections so concurrent calls do not share a s…
allen0099 Sep 5, 2026
ae33f44
feat(backends): add an atomic increment counter
allen0099 Sep 5, 2026
95f7bf7
feat(backends): add atomic get_and_delete and use it for one-shot values
allen0099 Sep 5, 2026
e570ce8
docs: describe the atomic backend primitives and the pooled Memcached…
allen0099 Sep 5, 2026
8edd29a
refactor(state): share the decode and peek logic between the state re…
allen0099 Sep 5, 2026
c50428a
test(state): cover peeking at a state past its wall-clock expiry
allen0099 Sep 5, 2026
2729cf0
refactor(routes): parse backend entries once for both monitoring views
allen0099 Sep 5, 2026
7df672e
test(routes): cover non-route keys and a missing backend
allen0099 Sep 5, 2026
f5bf823
refactor(cache): build Cache-Control once and share the ETag and 304 …
allen0099 Sep 5, 2026
face8ba
refactor(session): share the expiry, scan and save paths in SessionMa…
allen0099 Sep 5, 2026
4e68b65
test(session): cover scans skipping keys outside the session prefix
allen0099 Sep 5, 2026
7c9d918
feat(backends): add batched delete_many and use it for clear_prefix
allen0099 Sep 5, 2026
a476b94
refactor(manager): let json.loads decode the cached bytes itself
allen0099 Sep 5, 2026
30c3226
docs: mention delete_many behind clear_prefix
allen0099 Sep 5, 2026
ae8b32c
chore: bump version to 0.3.3
allen0099 Sep 5, 2026
06eac45
fix(memcached): wait for write acknowledgements on pooled connections
allen0099 Sep 5, 2026
5b93672
test(session): sort imports in session manager tests
allen0099 Sep 5, 2026
de1ed30
chore(deps): lock file maintenance
allen0099 Sep 5, 2026
1b8ac82
chore(ruff): ignore CPY001 and PLR0917 stabilized in ruff 0.16
allen0099 Sep 5, 2026
f1349cb
style(redis): join re-export import as ruff 0.16.6 sorts it
allen0099 Sep 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ The library has four independent subsystems:
- Keys live under their own `cache:`-prefixed namespace by default (configurable via `key_prefix`), separate from HTTP route keys and `oauth_state:`.
- `get()` never raises — returns `default` (`None` unless overridden) on a miss or decode failure. `set()` lets `TypeError` propagate for non-JSON-serializable values.
- `CacheManagerProxy` mirrors `BackendProxy`/`SessionManagerProxy`. The `AppCache` FastAPI dependency (`get_app_cache`, in `dependencies.py`) lazily creates and registers a default `CacheManager` on first use.
- `clear()`/`clear_prefix()` are built on `backend.get_all_keys()`, so they are no-ops on the Memcached backend (see below).
- `clear()`/`clear_prefix()` are built on `backend.get_all_keys()` + `backend.delete_many()`, so they are no-ops on the Memcached backend (see below).

**3. Session Management (`fastapi_cachex/session/`)**
- Optional subsystem, activated via `SessionMiddleware` and `SessionManagerProxy`.
Expand All @@ -80,10 +80,18 @@ The library has four independent subsystems:
All backends implement `BaseCacheBackend` (abstract base in `backends/base.py`):
- `MemoryBackend`: In-process dict with background cleanup task. Not suitable for multi-process production use.
- `AsyncRedisCacheBackend` (`backends/redis.py`): Fully async; uses `SCAN` (not `KEYS`) for pattern operations. Requires `redis[hiredis]` and `orjson` extras.
- `MemcachedBackend` (`backends/memcached.py`): `clear_pattern`/`get_all_keys` are no-ops (return `0`/`[]` with a `RuntimeWarning`) since the Memcached protocol has no key enumeration. Requires `pymemcache` extra.
- `MemcachedBackend` (`backends/memcached.py`): `clear_pattern`/`get_all_keys` are no-ops (return `0`/`[]` with a `RuntimeWarning`) since the Memcached protocol has no key enumeration. Runs the sync pymemcache client in worker threads with connection pooling (`use_pooling=True`, `default_noreply=False`), so concurrent calls never share a socket and every write is acknowledged before the next call on another socket can observe it. Requires `pymemcache` extra.

Backend keys are namespaced automatically (default prefix: `fastapi_cachex:`).

Two non-abstract atomic primitives live on the base class with non-atomic fallbacks, and every built-in backend overrides them (see README "Atomic backend primitives"):
- `increment(key, delta=1, ttl=None) -> int`: fixed-window counter; `ttl` applies only when the counter is created. Redis runs a registered Lua script, Memcached uses `ADD` + `INCR`/`DECR`, memory works under its lock. A counter reads back through `get()` as a `CacheEntry` with `COUNTER_FINGERPRINT` (`types.py`).
- `get_and_delete(key) -> CacheEntry | None`: one-shot retrieval (Redis `GETDEL`, Memcached get + `delete(noreply=False)` winner check). `StateManager.consume_state`, `delete_state`, `CacheManager.delete` and `invalidate()` use it. `delete()` keeps returning `None` for 0.3.x compatibility.

`delete_many(keys) -> int` is the third non-abstract base method: a per-key loop by default, one batched operation on Redis (`DEL`) and Memory (single lock).

`backends/codec.py` holds the JSON `CacheEntry` codec shared by Redis and Memcached; `decode_entry` maps a bare integer to a counter entry and every malformed value to `None`.

### Test Setup

`tests/conftest.py` sets `MemoryBackend` as the default backend via an `autouse=True` fixture for every test. Tests requiring Redis or Memcached must configure their own backends. The `memory_backend` fixture manages the cleanup task lifecycle.
Expand Down
42 changes: 41 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ cache and OAuth state, so `clear()`/`clear_prefix()` never touch unrelated cache
entries.

**Note**: `clear()`/`clear_prefix()` are implemented via the backend's
`get_all_keys()`. Since Memcached doesn't support key enumeration (see
`get_all_keys()` and `delete_many()` (one batched `DEL` on Redis). Since Memcached doesn't support key enumeration (see
[Memcached limitations](#memcached)), these two methods are no-ops on a
Memcached backend — `get()`/`set()`/`delete()`/`has()` work normally. Use
Redis or the in-memory backend if you need bulk clearing.
Expand Down Expand Up @@ -179,6 +179,41 @@ When a cached entry is valid (within TTL):

This means **cached hits are extremely fast** - the endpoint handler function is never executed.

### Atomic backend primitives

Every backend exposes two atomic operations on top of `get`/`set`/`delete`, for
values that are read and written by many concurrent requests:

```python
from fastapi_cachex import BackendProxy

backend = BackendProxy.get()

# Fixed-window counter: created on first use, `ttl` applies only then.
hits = await backend.increment(f"resend:{user_id}", ttl=86400)
if hits > 3:
raise TooManyRequests()

# One-shot value: of several concurrent callers exactly one gets the entry.
grant = await backend.get_and_delete(f"grant:{token}")
```

- `increment(key, delta=1, ttl=None) -> int` — Memory does the read-modify-write
under its lock, Redis runs a Lua script (`EXISTS` + `INCRBY` + `EXPIRE`) and
Memcached uses `ADD` + `INCR`/`DECR` (Memcached counters stop at 0). The
counter is visible through `get()` as a `CacheEntry` with fingerprint
`COUNTER_FINGERPRINT` and the decimal value as content, so `delete`/`clear*`
and the monitoring routes treat it like any other entry. Incrementing a key
that holds a cached response raises `CacheXError`.
- `get_and_delete(key) -> CacheEntry | None` — Memory pops under its lock, Redis
uses `GETDEL` (server 6.2+) and Memcached returns the value only when its own
`DELETE` won. `StateManager.consume_state`, `CacheManager.delete` and
`invalidate()` are built on it.

Both have a non-atomic fallback on `BaseCacheBackend`, so a third-party backend
that only implements the abstract methods keeps working; override them to get
real atomicity.

### In-Memory Cache (default)

If you don't specify a backend, FastAPI-CacheX will use the in-memory cache by default.
Expand Down Expand Up @@ -211,6 +246,11 @@ BackendProxy.set(backend)
- Keys are namespaced with `fastapi_cachex:` prefix to avoid conflicts
- Consider using Redis backend if you need pattern-based cache clearing

The synchronous pymemcache client runs in worker threads and is connection-pooled,
so concurrent requests never share a socket. Writes wait for the server's
acknowledgement (`default_noreply=False`), which keeps a value readable from
any pooled connection as soon as `set()` returns.

### Redis

```python
Expand Down
67 changes: 67 additions & 0 deletions fastapi_cachex/backends/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@

from abc import ABC
from abc import abstractmethod
from collections.abc import Iterable
from typing import Any

from fastapi_cachex.types import CacheEntry
from fastapi_cachex.types import counter_entry
from fastapi_cachex.types import counter_value


class BaseCacheBackend(ABC):
Expand All @@ -22,6 +25,70 @@ async def set(self, key: str, value: CacheEntry, ttl: int | None = None) -> None
async def delete(self, key: str) -> None:
"""Remove a response from the cache."""

async def delete_many(self, keys: Iterable[str]) -> int:
"""Remove every key in ``keys``; returns how many were removed.

The base implementation deletes one key at a time and reports how
many were attempted, since ``delete`` does not say whether the key
existed. The built-in backends override it with a single batched
operation that counts what was actually removed.
"""
count = 0
for key in keys:
await self.delete(key)
count += 1
return count

async def get_and_delete(self, key: str) -> CacheEntry | None:
"""Atomically retrieve and remove a cached entry.

Use this for one-shot values (OAuth states, grants, invalidation) where
exactly one of several concurrent callers may win: every other caller
sees ``None``.

The base implementation is a best-effort, NON-atomic get-then-delete
fallback for third-party backends; the built-in backends override it
with an atomic implementation.

Returns:
The entry that was stored under ``key``, or ``None`` if there was none
"""
value = await self.get(key)
if value is not None:
await self.delete(key)
return value

async def increment(self, key: str, delta: int = 1, ttl: int | None = None) -> int:
"""Atomically add ``delta`` to the integer counter stored at ``key``.

A missing key counts as 0: the first call creates the counter with the
value ``delta`` and applies ``ttl`` (seconds; ``None`` = never expires).
Later calls keep the existing expiry, so the counter lives in a fixed
window that starts when it is created - the shape rate limiters need.
The counter is readable through ``get()`` as a ``CacheEntry`` whose
fingerprint is ``COUNTER_FINGERPRINT`` and whose content is the decimal
value; ``delete``/``clear*`` treat it like any other entry.

The base implementation is a best-effort, NON-atomic read-modify-write
fallback for third-party backends and re-applies ``ttl`` on every call.
The built-in backends override it with a single server-side operation.

Args:
key: Cache key of the counter
delta: Amount to add (may be negative)
ttl: Time to live in seconds, applied when the counter is created

Returns:
The counter value after the increment

Raises:
CacheXError: If ``key`` holds a cached response instead of a counter
"""
current = await self.get(key)
value = delta if current is None else counter_value(current) + delta
await self.set(key, counter_entry(value), ttl=ttl)
return value

@abstractmethod
async def clear(self) -> None:
"""Clear all cached responses."""
Expand Down
69 changes: 69 additions & 0 deletions fastapi_cachex/backends/codec.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
"""Serialization shared by the network backends (Redis, Memcached).

Both backends store a ``CacheEntry`` as a JSON document; ``orjson`` is used when
it is installed and the standard library ``json`` module otherwise.
"""

from fastapi_cachex.types import CacheEntry
from fastapi_cachex.types import counter_entry

try:
import orjson as json

except ImportError: # pragma: no cover
import json # type: ignore[no-redef] # pragma: no cover

# ``json.loads`` (either implementation) raises ``ValueError`` subclasses for bad
# JSON; ``KeyError``/``TypeError``/``AttributeError`` cover documents whose shape
# is not the one ``encode_entry`` writes (missing fields, non-string content).
_DECODE_ERRORS = (ValueError, KeyError, TypeError, AttributeError)


def encode_entry(entry: CacheEntry) -> bytes:
"""Serialize a ``CacheEntry`` to a UTF-8 JSON document.

The raw content bytes are passed through ``latin-1`` so that arbitrary
bytes round-trip through JSON text.
"""
serialized: str | bytes = json.dumps(
{
"fingerprint": entry.fingerprint,
"content": entry.content.decode("latin-1"),
"media_type": entry.media_type,
},
)
# orjson returns bytes, stdlib json returns str
return serialized if isinstance(serialized, bytes) else serialized.encode("utf-8")


def _as_counter(raw: str | bytes) -> int | None:
"""The integer a bare counter value holds, or ``None`` for anything else."""
try:
return int(raw)
except ValueError:
return None


def decode_entry(raw: str | bytes | None) -> CacheEntry | None:
"""Rebuild a ``CacheEntry`` from a stored value.

A bare integer (what the server-side ``INCR`` family leaves behind for
``increment``) becomes a counter entry. Anything else that is not a document
written by ``encode_entry`` (corrupt JSON, missing fields, non-string
content) yields ``None``, so callers can treat every malformed value as a
cache miss.
"""
if raw is None:
return None
counter = _as_counter(raw)
if counter is not None:
return counter_entry(counter)
try:
data = json.loads(raw)
return CacheEntry(
fingerprint=data["fingerprint"],
content=data["content"].encode("latin-1"),
media_type=data.get("media_type"),
)
except _DECODE_ERRORS:
return None
Loading
Loading