diff --git a/content/en/real_user_monitoring/correlate_with_other_telemetry/apm/_index.md b/content/en/real_user_monitoring/correlate_with_other_telemetry/apm/_index.md index c725395d84d5a..b26411884677e 100644 --- a/content/en/real_user_monitoring/correlate_with_other_telemetry/apm/_index.md +++ b/content/en/real_user_monitoring/correlate_with_other_telemetry/apm/_index.md @@ -458,7 +458,7 @@ To verify you've configured the APM integration with RUM, follow the steps below To view traces from the RUM Explorer: -1. Navigate to your [list of sessions][21] and click on a session that has traces available. You can also query for sessions with traces by using`@_dd.trace_id:*`. +1. Navigate to your [list of sessions][22] and click on a session that has traces available. You can also query for sessions with traces by using`@_dd.trace_id:*`. When you select a session, the session panel appears with a request duration breakdown, a flame graph for each span, and a **View Trace in APM** link. @@ -711,15 +711,19 @@ Example for b3 multiple headers: These HTTP headers are not CORS-safelisted, so you need to [configure Access-Control-Allow-Headers][17] on your server handling requests that the SDK is set up to monitor. The server must also accept [preflight requests][18] (OPTIONS requests), which are made by the browser prior to every request when tracing is allowed on cross-site URLs. -## Effect on APM quotas +## Trace retention -Connecting RUM and traces may significantly increase the APM-ingested volumes. Use the initialization parameter `traceSampleRate` to keep a share of the backend traces starting from browser and mobile requests. +Ingested traces are available for 15 minutes in the [Live Search][19] explorer. To retain the traces for a longer period of time, [create APM retention filters][20]. Scope these retention filters on any span tag to retain traces for critical pages and user actions. -## Trace retention +If using RUM Without Limits, you can also use [cross-product retention filters][21] to retain APM traces associated to specific RUM sessions, optimizing the correlation between your frontend and your backend. By default 1% of RUM [sessions and their traces are automatically retained][23] at no additional cost. + +## Effect on APM quotas + +Connecting RUM and traces may significantly increase the APM-ingested volumes. Use the initialization parameter `traceSampleRate` to control a share of the backend traces starting from browser and mobile requests to ingest. -These traces are available for 15 minutes in the [Live Search][19] explorer. To retain the traces for a longer period of time, create [retention filters][20]. Scope these retention filters on any span tag to retain traces for critical pages and user actions. +Configuring cross-product retention filters may also increase the APM-indexed volumes. Use the retention rate of the cross-product retention filters to control the share of the backend traces to index. -## Further Reading +## Further reading {{< partial name="whats-next/whats-next.html" >}} @@ -743,4 +747,6 @@ These traces are available for 15 minutes in the [Live Search][19] explorer. To [18]: https://developer.mozilla.org/en-US/docs/Glossary/Preflight_request [19]: /tracing/trace_explorer/#live-search-for-15-minutes [20]: /tracing/trace_pipeline/trace_retention/#retention-filters -[21]: https://app.datadoghq.com/rum/explorer +[21]: /real_user_monitoring/rum_without_limits/retention_filters/#cross-product-retention-filters +[22]: https://app.datadoghq.com/rum/explorer +[23]: /tracing/trace_pipeline/trace_retention/#one-percent-flat-sampling diff --git a/content/en/real_user_monitoring/rum_without_limits/retention_filters.md b/content/en/real_user_monitoring/rum_without_limits/retention_filters.md index 87071c3f59ae9..6cf997f7a02b3 100644 --- a/content/en/real_user_monitoring/rum_without_limits/retention_filters.md +++ b/content/en/real_user_monitoring/rum_without_limits/retention_filters.md @@ -36,7 +36,7 @@ The logical flow of retention filters is the following: - Be cautious when defining filters on event attributes that update over time. For example, a filter retaining sessions with fewer than two errors might mistakenly retain sessions, as error counts update in real-time, and all sessions start at zero. Either use "greater than or equal to" (≥) conditions for fields that update, such as `@session.error.count >= 2`, or ensure the Session and View objects that are mutable are complete before evaluating them against the retention filters, by adding `@session.is_active: false` or `@view.is_active: false`. - Our SDKs batch and compress events before sending them to Datadog, and failed uploads go back at the end of the queue on the device. Therefore, it could happen that event `B` is evaluated before event `A`, but all events are eventually evaluated against the list of retention filters to prevent gaps. -## How retention filters work with replays + ## Creating a retention filter @@ -109,6 +109,45 @@ For example, to exclude sessions from South Korea while retaining all other sess **Note**: There is no way to prevent a specific event from being retained. You can use negative queries (for instance, adding `-@error.message:"Script error."` to a retention filter targeting RUM Errors) to minimize the volume of undesired events, but other retention filters may still make a positive retention decision about a session that contains the event you tried to filter out. +## Cross-product retention filters + +{{< callout url= + btn_hidden="false" header="Join the Preview!">}} +Cross-Product Retention Filters are in Preview. Use this form to submit your request today. +{{< /callout >}} + +When configuring a RUM retention filter, you can enable two cross-product retention filters: one for session replays and one for APM traces. + +- **Session Replay filter**: Retains replays for the specified percentage of sessions retained by the parent RUM retention filter which have an available replay. +- **APM traces filter**: Indexes APM traces for the specified percentage of sessions retained by the parent RUM retention filter which have available traces. +
The APM traces filter is only compatible with the following versions of the SDKs (applies to RUM and APM, but not Session Replay):
- Browser 6.5.0+
- Android 3.0.0+
- iOS 3.3.0+
- React Native 3.0.0+
+ +
Configuring cross-product retention filters may increase APM-indexed volumes.
+ +**Note**: The availability of a session replay or APM traces depends on the initialization parameters `sessionReplaySampleRate` and `traceSampleRate` of the SDK. + +The cross-product retention filters allow you to optimize the correlation between different products to retain richer telemetry. + +### Example + +Consider a RUM retention filter configured as follows: + +- **RUM retention filter**: 60% - `@type:error` (retention filter retaining 60% of sessions with at least one error) +- **Cross-product retention filter Session Replay**: 50% +- **Cross-product retention filter APM traces**: 25% + +If across the sessions, 30% of them have an available replay and 40% of them have available traces, then the outcome is the following: + +- 60% of sessions with at least one error are retained +- 50% × 30% = 15% of these retained sessions have a replay +- 25% × 40% = 10% of these retained sessions have APM traces + +### Default 1% retention + +For compatible SDKs (see above), Datadog provides a default RUM retention filter and cross-product retention filter on APM traces that retains 1% of the sessions with available traces and their traces, at no additional cost. + +This default filter ensures that you always have a baseline of correlated APM data available for your RUM sessions, even before custom cross-product retention filters. + ## Best practices See [Retention Filter Best Practices][4]. diff --git a/content/en/tracing/trace_explorer/trace_queries.md b/content/en/tracing/trace_explorer/trace_queries.md index a8088d59a6161..4dfe1575a9e41 100644 --- a/content/en/tracing/trace_explorer/trace_queries.md +++ b/content/en/tracing/trace_explorer/trace_queries.md @@ -106,7 +106,7 @@ Trace Queries run on traces indexed by the [intelligent retention filter][3] and {{< img src="tracing/trace_queries/trace_queries_base_data.png" style="width:100%; background:none; border:none; box-shadow:none;" alt="Flow showing where trace retention filters apply in the processing pipeline" >}} The intelligent retention filter is enabled by default and includes: -- [Flat sampling][4]: Uniformly samples 1% of ingested spans. +- [Flat sampling][4]: Retains all traces associated with 1% of ingested RUM sessions, plus a uniform 1% sample of ingested spans, ensuring correlation between frontend sessions and backend traces. - [Diversity sampling][5]: Retains a diverse set of traces to maintain visibility across environments, services, operations, and resources. Both Flat sampling and Diversity sampling capture **complete traces**, meaning all spans within a trace are indexed to ensure accurate results in Trace Queries. diff --git a/content/en/tracing/trace_pipeline/trace_retention.md b/content/en/tracing/trace_pipeline/trace_retention.md index ac01f1d47a32d..d0d824f476a2f 100644 --- a/content/en/tracing/trace_pipeline/trace_retention.md +++ b/content/en/tracing/trace_pipeline/trace_retention.md @@ -107,7 +107,9 @@ The set of data captured by diversity sampling is not uniformly sampled (that is #### One percent flat sampling -The flat 1% sampling is a **uniform 1% sample** of [ingested spans][12]. It is applied based on the `trace_id`, meaning that all spans belonging to the same trace share the same sampling decision. +The flat 1% sampling captures: +1. All **traces associated with 1% of ingested RUM sessions**, ensuring all indexed sessions have associated trace data. This improves [correlation between APM and RUM][20], allowing you to debug user issues by viewing both frontend sessions and backend traces together. The sample is applied based on the `session_id`, meaning all traces linked to the same RUM session share a consistent indexing decision. +2. A **uniform 1% sample** of [ingested spans][12], applied based on the `trace_id` so all spans in the same trace share the same sampling decision. Use this sample for general system health monitoring and trend analysis. This sampling mechanism is uniform, and it is proportionally representative of the full ingested traffic. As a result, low-traffic services and endpoints might be missing from that dataset if you filter on a short time frame. @@ -161,7 +163,9 @@ The `retained_by` attribute is present on all retained spans. Its value is: - `retained_by:retention_filter` if the span was captured by a [custom retention filter](#create-your-own-retention-filter), including the [default retention filters](#default-retention-filters) and **no trace rate** was configured. These spans are not included in Trace Queries as trace queries require all spans of a trace to be indexed. - `retained_by:trace_retention_filter` if the span is captured by a retention filter for which a trace rate was configured. - `retained_by:diversity_sampling` if the span was captured by [diversity sampling](#diversity-sampling) (part of the [Intelligent retention filter](#datadog-intelligent-retention-filter)). -- `retained_by:flat_sampled` if the span was indexed by the [1% flat sampling](#one-percent-flat-sampling). +- `retained_by:flat_sampled` if the span was indexed by the [1% flat sampling](#one-percent-flat-sampling). Filter further by retention reason: + - `@retention_reason:rum` for traces linked to RUM sessions sampled based on the `session_id`. Use this to analyze traces correlated with user sessions. + - `@retention_reason:apm` for traces sampled uniformly based on the `trace_id`. Use this for general performance trends and system-wide analysis. {{< img src="tracing/trace_indexing_and_ingestion/retention_filters/trace_analytics.png" style="width:100%;" alt="Retained By facet" >}} @@ -192,3 +196,4 @@ Spans indexed by the intelligent retention filter are **excluded** from APM trac [17]: /dynamic_instrumentation/ [18]: https://app.datadoghq.com/apm/traces/retention-filters [19]: /monitors/types/apm/?tab=traceanalytics +[20]: /tracing/other_telemetry/rum/ \ No newline at end of file