Skip to content

Support async cache clients in FlagDefinitionCacheProvider #567

@nachogarcia

Description

@nachogarcia

Problem Statement

Context

We use posthog-python for server-side local evaluation behind a FastAPI app
running on Azure Container Apps. Multiple worker processes share a Redis
instance and we use a FlagDefinitionCacheProvider (modeled on
examples/redis_flag_cache.py) so only one worker polls PostHog at a time.

The rest of our codebase is async-first — every Redis call is redis.asyncio.
The FlagDefinitionCacheProvider protocol is sync, so to plug into it we had
to build a sync facade that owns a dedicated daemon thread and its own event
loop, then submit coroutines via run_coroutine_threadsafe for every call.
That's ~80 lines of plumbing solely to bridge redis.asyncio into a sync
contract whose only caller is itself a background thread.

Request

Allow async cache providers — i.e. let get_flag_definitions,
should_fetch_flag_definitions, on_flag_definitions_received, and shutdown
return awaitables that the SDK runs to completion before continuing.

Because the provider is only invoked from _load_feature_flags on the
Poller daemon thread (not from any synchronous evaluation path), the SDK
can transparently support this by running an event loop on that thread —
no API break, no opt-in flag needed.

Solution Brainstorm

Suggested shape

Option A — single protocol, awaitable-aware:

class FlagDefinitionCacheProvider(Protocol):
    def get_flag_definitions(self) -> (
        Optional[FlagDefinitionCacheData] | Awaitable[Optional[FlagDefinitionCacheData]]
    ): ...
    # ... etc

In _load_feature_flags, wrap each call:

result = provider.get_flag_definitions()
if inspect.isawaitable(result):
    result = _run_on_poller_loop(result)

with _run_on_poller_loop lazily creating a single asyncio event loop bound
to the polling thread (created once, reused across ticks).

Option B — separate AsyncFlagDefinitionCacheProvider protocol, accepted by
the same flag_definition_cache_provider= kwarg with isinstance dispatch.
Less elegant but explicit.

Option A keeps the public surface unchanged and lets users pass async
providers without thinking about it.

What we'd contribute

Happy to send a PR implementing Option A with:

Want to confirm the direction before opening the PR — happy to do Option B
instead if you'd rather keep the sync protocol strictly sync.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions