4.14.0rc1
Pre-release
Pre-release
Release Notes
New Features
- AI Guard: Adds a redaction feature that can be enabled in the UI. For more information, see https://docs.datadoghq.com/security/ai_guard/setup/sdk/.
- CI Visibility: Adds a new
ci.pipeline.display_nametag for the Buildkite provider, populated from theBUILDKITE_PIPELINE_NAMEenvironment variable.
- Data Streams Monitoring: Adds an optional
tagsargument toddtrace.data_streams.set_produce_checkpointandddtrace.data_streams.set_consume_checkpoint. Pass a list of"key:value"strings to attach additional edge tags to a manual checkpoint alongside the automatically settype,topic, anddirectiontags—for example,set_produce_checkpoint("eventbridge", detail_type, carrier_set, tags=["exchange:my-bus"]). Existing calls that do not passtagsare unaffected.
- LLM Observability: Adds
LLMObs.list_experiments()to query experiments by logical name and metadata containment (for example,{"tags": ["git.commit.sha:abc123"]}). It returns run status and aggregate data (evaluations, token costs, and error rates), allowing CI/CD workflows to compare a run against a baseline commit and gate on the result.
- LLM Observability: Adds support for manually versioning agents. Pass
versionwhen starting an agent span, either withLLMObs.agent(name="my-agent", version="v3")or with the@agent(version="v3")decorator. For agents created by an auto-instrumented framework, useLLMObs.annotation_context(agent={"version": "v3"}). This sets theagent_versiontag on agent spans. Theagentargument accepts either a plain dictionary or the newddtrace.llmobs.AgentTypedDictclass.
- LLM Observability: Adds
LLMObs.submit_feedback()for submitting end-user feedback associated with a span, trace, session, or customer-defined feedback join key.
- openfeature: Adds agentless delivery for the Feature Flagging and Experimentation (FFE) OpenFeature provider, loading Universal Flag Configuration directly from Datadog over HTTPS without requiring a Datadog Agent. The agentless source is now the default and requires
DD_API_KEY; setDD_FEATURE_FLAGS_CONFIGURATION_SOURCE=remote_configto continue loading configuration through the Datadog Agent's Remote Configuration.
- opentelemetry: OTLP trace export now supports the
http/protobufprotocol (the default) andhttp/json, selected via theOTEL_EXPORTER_OTLP_TRACES_PROTOCOLenvironment variable.
- tracing: Adds
Span.remove_tag(key)andSpan.remove_metric(key)for removing a previously set tag or metric from a span. Both are no-ops if the key is not set.
Bug Fixes
- AAP: Fixes incorrect request fingerprints when a web application is wrapped by same-service tracing middleware.
- botocore: Fixes an issue where LLM Observability spans for Bedrock
invoke_model_with_response_streamcalls reported all token usage metrics as0. Token counts are now extracted from the streamed response and set on the span.
- CI Visibility: Fixes an issue where code coverage collected for Intelligent Test Runner (ITR) included files from
site-packagesordist-packagesdirectories.
- CI Visibility: Fixes an issue where the session status was reported as
passinstead offailwhen pytest exited with a non-OK, non-NO_TESTS_COLLECTEDexit code (for example, because of collection errors, interruptions, or internal errors).
- Code Security: Fixes an issue where applications could crash when asynchronous tasks accessed request data after the originating request completed.
- django: Fixes an issue (#19454) where Django processes that never serve HTTP requests, such as Celery and Dramatiq workers, used substantially more memory and took longer to start. API endpoint discovery now runs when the WSGI or ASGI application is built rather than during
django.setup().
- gevent: Fixes an issue where running an application with
ddtrace-run(or single-step instrumentation) could raise aPydanticSchemaGenerationErrorat import time for models that use forward references whengeventis installed and the application runs on CPython 3.14 or later. Theannotationlibmodule, which now owns theForwardRefclass on CPython 3.14, is preserved during module cloning so thatpydanticresolves forward references correctly.
- kombu: Fixes an issue where messages published to the RabbitMQ default (unnamed) exchange were not linked to their consumers in the Data Streams Monitoring map.
- OpenTelemetry logs: Fixes an issue where enabling OpenTelemetry logs collection (
DD_LOGS_OTEL_ENABLED=true) could cause the tracer to export its own internal log records, producing a growing volume of exported logs. The tracer's own log records and the OpenTelemetry exporter's log records are now excluded from OpenTelemetry logs export, while application logs continue to be collected and exported as before.
- profiling: Fixes an issue where applications using uWSGI with
--lazy-appsand--skip-atexitcould crash while workers shut down on Python 3.12 or later.
- profiling: Fixes an issue where child spans created from propagated tracing contexts could lose local-root and endpoint attribution when greenlets shared a thread.
- pytorch: Fixes an issue where the
pytorch.rankspan could end too soon.
- requests: Fixes an issue where distributed tracing context propagated on outbound
requestscalls pointed to the wrong span when theurllib3integration was also enabled (for example, viaDD_TRACE_URLLIB3_ENABLED=true), causing those requests to be missing or to show incorrect latency in downstream traces.
- tracing: Fixes an issue where the sampling decision of an incoming distributed trace was discarded, reported as a propagation error on the local root span, and logged as a warning when the trace was sampled by a mechanism unknown to this version of the library. Well-formed sampling decisions are now propagated unchanged.
- tracing: Fixes an issue where OpenTelemetry thread-context record publication could not be disabled on Linux. Publication can now be disabled by setting the experimental
DD_TRACE_OTEL_CTX_ENABLEDenvironment variable tofalse.
- tracing: Fixes an issue where OTLP trace export failures reported the Datadog Agent intake URL instead of the configured OTLP endpoint.
- tracing: Fixes an issue where
_dd.svc_srcwas incorrectly set to either"m"or the service name.
Other Changes
- botocore: Payload-tagging redaction now uses
python-jsonpathin place of the vendoredjsonpath-ngandply, soddtraceno longer shipsply, which is unmaintained and flagged by dependency scanners forCVE-2025-56005. Existing redaction paths continue to work.