4.14.0

Estimated end-of-life date, accurate to within three months: 05-2027

See the support level definitions for more information.

New Features

• AI Guard:

  • Adds a redaction feature that can be enabled in the UI. For more information, see the AI Guard SDK setup documentation.

• CI Visibility:

  • Adds a new ci.pipeline.display_name tag for the Buildkite provider, populated from the BUILDKITE_PIPELINE_NAME environment variable.

• Data Streams Monitoring:

  • Adds an optional tags argument to ddtrace.data_streams.set_produce_checkpoint and ddtrace.data_streams.set_consume_checkpoint. Pass a list of "key:value" strings to attach additional edge tags to a manual checkpoint alongside the automatically set type, topic, and direction tags. For example: set_produce_checkpoint("eventbridge", detail_type, carrier_set, tags=["exchange:my-bus"]). Existing calls that do not pass tags are unaffected.

• LLM Observability:

  • Adds LLMObs.list_experiments() to query experiments by logical name and metadata containment, such as {"tags": ["git.commit.sha:abc123"]}. It returns run status and aggregate data—including evaluations, token costs, and error rates—so CI/CD workflows can compare a run against a baseline commit and gate on the result.
  • Adds support for manually versioning agents. Pass version when starting an agent span, either with LLMObs.agent(name="my-agent", version="v3") or with the @agent(version="v3") decorator. For agents created by an auto-instrumented framework, use LLMObs.annotation_context(agent={"version": "v3"}). This sets the agent_version tag on agent spans. The agent argument accepts a plain dictionary or the new ddtrace.llmobs.Agent TypedDict class.
  • Adds LLMObs.submit_feedback() for submitting end-user feedback associated with a span, trace, session, or customer-defined feedback join key.

• OpenFeature:

  • Adds agentless delivery for the Feature Flagging and Experimentation (FFE) OpenFeature provider, loading Universal Flag Configuration directly from Datadog over HTTPS without requiring a Datadog Agent. Agentless delivery is now the default and requires DD_API_KEY. Set DD_FEATURE_FLAGS_CONFIGURATION_SOURCE=remote_config to continue loading configuration through the Datadog Agent's Remote Configuration.

• OpenTelemetry:

  • OTLP trace export now supports the http/protobuf protocol—the default—and http/json, selected through the OTEL_EXPORTER_OTLP_TRACES_PROTOCOL environment variable.

• Tracing:

  • Adds Span.remove_tag(key) and Span.remove_metric(key) for removing a previously set tag or metric from a span. Both methods are no-ops if the key is not set.

Bug Fixes

• AAP:

  • Fixes incorrect request fingerprints when a web application is wrapped by same-service tracing middleware.

• Botocore:

  • Fixes an issue where LLM Observability spans for Bedrock invoke_model_with_response_stream calls reported all token usage metrics as 0. Token counts are now extracted from the streamed response and set on the span.

• CI Visibility:

  • Fixes an issue where code coverage collected for Intelligent Test Runner included files from site-packages or dist-packages directories.
  • Fixes an issue where the session status was reported as pass instead of fail when pytest exited with a non-OK, non-NO_TESTS_COLLECTED exit code, such as for collection errors, interruptions, or internal errors.

• Code Origin for Spans:

  • Prevents views or traced functions from raising an exception when a large number of them are present.

• Code Security:

  • Fixes an issue where applications could crash when asynchronous tasks accessed request data after the originating request completed.

• Django:

  • Fixes an issue where Django processes that never serve HTTP requests, such as Celery and Dramatiq workers, used substantially more memory and took longer to start. API endpoint discovery now runs when the WSGI or ASGI application is built rather than during django.setup().

• Gevent:

  • Fixes an issue where running an application with ddtrace-run or single-step instrumentation could raise a PydanticSchemaGenerationError at import time for models that use forward references when gevent is installed and the application runs on CPython 3.14 or later. The annotationlib module, which now owns the ForwardRef class on CPython 3.14, is preserved during module cloning so that Pydantic resolves forward references correctly.

• Kombu:

  • Fixes an issue where messages published to the RabbitMQ default unnamed exchange were not linked to their consumers in the Data Streams Monitoring map.

• OpenTelemetry Logs:

  • Fixes an issue where enabling OpenTelemetry logs collection with DD_LOGS_OTEL_ENABLED=true could cause the tracer to export its own internal log records, producing a growing volume of exported logs. The tracer's own log records and the OpenTelemetry exporter's log records are now excluded, while application logs continue to be collected and exported.

• Profiling:

  • Fixes a rare crash when using gevent.
  • Fixes an issue where applications using uWSGI with --lazy-apps and --skip-atexit could crash while workers shut down on Python 3.12 or later.
  • Fixes an issue where child spans created from propagated tracing contexts could lose local-root and endpoint attribution when greenlets shared a thread.

• PyTorch:

  • Fixes an issue where the pytorch.rank span could end too soon.

• Requests:

  • Fixes an issue where distributed tracing context propagated on outbound requests calls pointed to the wrong span when the urllib3 integration was also enabled—for example, through DD_TRACE_URLLIB3_ENABLED=true—causing those requests to be missing or to show incorrect latency in downstream traces.

• Snowflake:

  • Fixes an issue where traced cursor calls failed when command was passed as a keyword argument to execute or executemany.

• Tracing:

  • Fixes an issue where the sampling decision of an incoming distributed trace was discarded, reported as a propagation error on the local root span, and logged as a warning when the trace was sampled by a mechanism unknown to this version of the library. Well-formed sampling decisions are now propagated unchanged.
  • Fixes an issue where OpenTelemetry thread-context record publication could not be disabled on Linux. Publication can now be disabled by setting the experimental DD_TRACE_OTEL_CTX_ENABLED environment variable to false.
  • Fixes an issue where OTLP trace export failures reported the Datadog Agent intake URL instead of the configured OTLP endpoint.
  • Fixes an issue where _dd.svc_src was incorrectly set to either m or the service name.

Other Changes

• Botocore:

  • Payload-tagging redaction now uses python-jsonpath instead of the vendored jsonpath-ng and ply. As a result, ddtrace no longer ships ply, which is unmaintained and reported by dependency scanners as CVE-2025-56005. Existing redaction paths continue to work.