feat(dramatiq): add dramatiq integration

[erikayasuda]

Overview

This PR adds the dramatiq library as a supported integration for dd-trace-py. This addresses #5043.

In Scope for this PR

• Instrumenting the dramatiq.actor.Actor.send_with_options() method. This is the method called when a function with the @dramatiq.actor decorator is called asynchronously. See example below:

# app.py
import dramatiq
from flask import Flask

app = Flask(__name__)

@dramatiq.actor
def my_func():
   return "response"

@dramatiq.actor
def my_other_func(a: int, b: int) -> int:
   return a + b

@app.route('/')
def index():
   my_func.send() # this calls send_with_options() under the hood
   my_other_func.send_with_options(args=(1, 1), options={"max_retries": 3})
   return 'hello world'

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=6060)

Running the above example Flask app with ddtrace-run flask run and hitting the / endpoint would generate a trace in the UI like the following:

The detailed span info for my_other_func.send_with_options(...) would look something like this:

{
   actor: {
      name: my_other_func
      options: {"max_retries": 3}
   }
   env: test
   language: python
   span: {
      kind: producer
   }
}

NOTE

• The duration of the span is of the send_with_options() call itself, and not reflective of the execution duration of the function being asynchronously completed.
• When calling a function asynchronously with send(), it will also display on the UI as a send_with_options() span. This is because send() calls send_with_options() with empty options, so tracing both of these functions would create two spans for every instance that send() is called, with the exact same span information.

Out of Scope for this PR

• All other dramatiq methods.
• Supporting the actual duration of the function executed asynchronously by send_with_options()

The above and additional features can be investigated and added to this initial iteration at a later time.

Checklist

• Change(s) are motivated and described in the PR description
• Testing strategy is described if automated tests are not included in the PR
• Risks are described (performance impact, potential for breakage, maintainability)
• Change is maintainable (easy to change, telemetry, documentation)
• Library release note guidelines are followed or label changelog/no-changelog is set
• Documentation is included (in-code, generated user docs, public corp docs)
• Backport labels are set (if applicable)
• If this PR changes the public interface, I've notified @DataDog/apm-tees.
• If change touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.

Reviewer Checklist

• Title is accurate
• All changes are related to the pull request's stated goal
• Description motivates each change
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Change is maintainable (easy to change, telemetry, documentation)
• Release note makes sense to a user of the library
• Author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[pr-commenter]

Benchmarks

Benchmark execution time: 2024-03-27 19:03:12

Comparing candidate commit 3dae826 in PR branch erikayasuda/dramatiq-integration with baseline commit 141b09b in branch main.

Found 1 performance improvements and 6 performance regressions! Performance is the same for 194 metrics, 9 unstable metrics.

scenario:httppropagationextract-invalid_span_id_header

• 🟥 max_rss_usage [+575.295KB; +723.546KB] or [+2.713%; +3.412%]

scenario:httppropagationextract-tracecontext_headers

• 🟥 max_rss_usage [+488.502KB; +687.460KB] or [+2.300%; +3.237%]

scenario:httppropagationextract-wsgi_valid_headers_all

• 🟥 max_rss_usage [+494.865KB; +742.946KB] or [+2.328%; +3.495%]

scenario:sethttpmeta-all-disabled

• 🟥 max_rss_usage [+515.342KB; +744.587KB] or [+2.394%; +3.459%]

scenario:sethttpmeta-obfuscation-send-querystring-disabled

• 🟩 max_rss_usage [-647.578KB; -452.608KB] or [-2.901%; -2.028%]

scenario:span-start-finish-telemetry

• 🟥 max_rss_usage [+593.318KB; +757.133KB] or [+2.791%; +3.561%]

scenario:span-start-finish-traceid128

• 🟥 max_rss_usage [+706.664KB; +779.365KB] or [+3.334%; +3.677%]

[datadog-dd-trace-py-rkomorn]

Datadog Report

Branch report: erikayasuda/dramatiq-integration
Commit report: cc3bde9
Test service: dd-trace-py

✅ 0 Failed, 1579 Passed, 1554 Skipped, 48m 28.85s Total duration (36m 29.33s time saved)