fix(futures): fix incorrect context propgation with ThreadPoolExecutor

[brettlangdon]

There is a bug when scheduling work onto a ThreadPoolExecutor and not waiting for the response (e.g. pool.submit(work), and ignoring the future) we not properly associate the spans created in the task with the trace that was active when submitting the task.

The reason for this bug is because we propagate the currently active span (parent) to the child task, however, if the parent span finishes before the child task can create it's first span, we no longer consider the parent span active/available to inherit from. This is because our context management code does not work if passing spans between thread or process boundaries.

The solution is to instead pass the active span's Context to the child task. This is a similar process as passing context between two services/processes via HTTP headers (for example). This change will allow the child task's spans to be properly associated with the parent span regardless of the execution order.

This issue can be highlighted by the following example:

pool = ThreadPoolExecutor(max_workers=1)

def task():
    parent_span = tracer.current_span()
    assert parent_span is not None
    time.sleep(1)


with tracer.trace("parent"):
    for _ in range(10):
        pool.submit(task)

The first execution of task will (probably) succeed without any issues because the parent span is likely still active at that time. However, when each additional task executes the assertion will fail because the parent span is no longer an active span so tracer.current_span() will return None.

This example shows that only the first execution of task will be properly associated with the parent span/trace, the other calls to task will be disconnected traces.

This fix will resolve this inconsistent and unexpected behavior to ensure that the spans created in task will always be properly associated with the parent span/trace.

This change may impact people who were expecting to access the current span in the child task, but before creating any spans in the child task (the code sample above), as the span will no longer be available via tracer.current_span().

Checklist

• Change(s) are motivated and described in the PR description
• Testing strategy is described if automated tests are not included in the PR
• Risks are described (performance impact, potential for breakage, maintainability)
• Change is maintainable (easy to change, telemetry, documentation)
• Library release note guidelines are followed or label changelog/no-changelog is set
• Documentation is included (in-code, generated user docs, public corp docs)
• Backport labels are set (if applicable)
• If this PR changes the public interface, I've notified @DataDog/apm-tees.

Reviewer Checklist

• Title is accurate
• All changes are related to the pull request's stated goal
• Description motivates each change
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Change is maintainable (easy to change, telemetry, documentation)
• Release note makes sense to a user of the library
• Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-06-20 18:20:37

Comparing candidate commit 5b8e7e5 in PR branch brettlangdon/fix.futures.propagation with baseline commit 0b1e1ce in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 187 metrics, 9 unstable metrics.

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: brettlangdon/fix.futures.propagation
Commit report: cca48d6
Test service: dd-trace-py

✅ 0 Failed, 576 Passed, 330 Skipped, 2m 44.42s Total duration (1m 4.24s time saved)