Skip to content

Task Processor: Propagate structlog contextvars across task runs #213

@khvn26

Description

@khvn26

Task handlers emit structured events via structlog, but each handler today has to carry its identifiers (organisation id, project id, user id, etc.) as explicit kwargs at every call site. Consumers like flagsmith/flagsmith want to bind those identifiers once at request time (via Django middleware) and have every downstream emit pick them up — including emits inside tasks enqueued during the request.

Task.trace_context already serves this propagation pattern for OpenTelemetry's W3C TraceContext. The same mechanism can carry structlog contextvars across the enqueue → run boundary, so that an event emitted from inside a task inherits the caller's bound context.

Proposal

In the task processor:

  • On enqueue, capture the current structlog.contextvars bindings (via structlog.contextvars.get_contextvars()) and serialise them into Task.trace_context under a dedicated subkey (e.g. structlog_context). This sits alongside the existing W3C propagation payload; OTel's propagate.extract() ignores unknown keys, so no conflict.
  • On run, before invoking the task handler, read trace_context["structlog_context"] and pass it to bind_contextvars(**ctx).
  • On completion (success or failure), clear_contextvars() in a finally block, so one task's bindings never leak into the next task run on the same worker thread.

Out of scope

  • Flagsmith-specific attribute names (organisation.id, user.id, etc.). The library is agnostic — it serialises whatever the caller has bound.
  • Middleware that binds the context in the first place. That's the consumer's concern; see the linked flagsmith/flagsmith issue.

Acceptance

  • Enqueueing a task while structlog contextvars are bound persists those bindings onto the task record.
  • Running that task restores the bindings for the handler's execution.
  • Bindings are cleared after the handler returns or raises — verified by running two tasks in sequence on the same worker, asserting the second doesn't inherit the first's state.
  • Existing OTel trace-context propagation keeps working (no regression on task.trace_context shape).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions