0.17.0 - The Cluster Strikes Back

This is a big one!

We finally have Redis cluster support thanks to some initial research and exploration by @zzstoatzz, who helped me realize that we could in fact do it! Point your Docket to a redis+cluster://... or rediss+cluster:// URL and everything should just work 🙏.

Second, we have a new kind of dependency function called a Shared dependency, and the idea is that these are shared across all your tasks for the lifespan of the Worker. Think FastAPI lifespan functions, but you declare them on your tasks instead of separately outside of them. It looks like this:

from contextlib import asynccontextmanager

from docket import Depends, Shared, task


async def load_config():
    """Worker-scoped: loaded once at startup, same config for all tasks."""
    return await fetch_config_from_vault()


@asynccontextmanager
async def db_pool():
    """Worker-scoped: created once, shared by all tasks, closed at shutdown."""
    pool = await create_pool("postgresql://...")
    yield pool
    await pool.close()


@asynccontextmanager
async def db_connection(pool: Pool = Shared(db_pool)):
    """Task-scoped: fresh connection per task, returned to pool after."""
    async with pool.connection() as conn:
        yield conn


@task
async def process_order(
    order_id: int,
    db: Connection = Depends(db_connection),
    config: Config = Shared(load_config),
):
    await db.execute(
        "UPDATE orders SET status = $1 WHERE id = $2",
        config.default_status,
        order_id,
    )

Big thanks to @omikader for the feature request and brainstorming that brought us to this.

What's Changed

• Add loq for file size enforcement by @chrisguidry in #277
• Add ACL test variants for Redis and Valkey by @chrisguidry in #278
• Split large test files under 500 LOC limit by @chrisguidry in #279
• Refactor large source files with mixins and extractions by @chrisguidry in #280
• Clean up stale artifacts before Pages deploy by @chrisguidry in #281
• Add Redis connection helpers to _redis.py by @chrisguidry in #283
• Add Redis Cluster support by @chrisguidry in #284
• Fix Dependabot security alerts for urllib3 and starlette by @chrisguidry in #286
• Add Shared dependencies for worker-scoped resources by @chrisguidry in #287
• Fix internal Redis instrumentation suppression (#269) by @chrisguidry in #289
• Remove OTel SDK from core dependencies by @chrisguidry in #291
• Move concurrency limiting from Worker to ConcurrencyLimit dependency by @chrisguidry in #293

Full Changelog: 0.16.6...0.17.0