-
Notifications
You must be signed in to change notification settings - Fork 1.2k
[DOCS-12593] RUM x APM retention filters update #32912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rtrieu
wants to merge
5
commits into
master
Choose a base branch
from
rtrieu/docs-12593-rum-apm-sampling-update
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+62
−12
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,15 +36,15 @@ The logical flow of retention filters is the following: | |
| - Be cautious when defining filters on event attributes that update over time. For example, a filter retaining sessions with fewer than two errors might mistakenly retain sessions, as error counts update in real-time, and all sessions start at zero. Either use "greater than or equal to" (≥) conditions for fields that update, such as `@session.error.count >= 2`, or ensure the Session and View objects that are mutable are complete before evaluating them against the retention filters, by adding `@session.is_active: false` or `@view.is_active: false`. | ||
| - Our SDKs batch and compress events before sending them to Datadog, and failed uploads go back at the end of the queue on the device. Therefore, it could happen that event `B` is evaluated before event `A`, but all events are eventually evaluated against the list of retention filters to prevent gaps. | ||
|
|
||
| ## How retention filters work with replays | ||
| <!-- ## How retention filters work with replays | ||
|
|
||
| You can manage session sampling with replays using retention filters. Whenever a session with replays is billed, both the session events and the video recording are kept and billed. This means that if you collect 100% of sessions and 100% of replays from SDKs, whenever a retention filter keeps a session, Datadog keeps and charges for both the session and the replay. | ||
|
|
||
| Replays collected through the [force collection][1] mechanism are kept by the default retention filter, positioned first in the list (see below). | ||
|
|
||
| {{< img src="real_user_monitoring/rum_without_limits/retention-session-filter.png" alt="When force collection is enabled, it is positioned first in the list of retention filters." style="width:90%" >}} | ||
|
|
||
| **Note**: Though Datadog's mobile SDKs also provide APIs to conditionally start and stop the recording (instead of relying on a flat sample rate), only the replays that are force-recorded by the Browser SDK are kept by default. | ||
| **Note**: Though Datadog's mobile SDKs also provide APIs to conditionally start and stop the recording (instead of relying on a flat sample rate), only the replays that are force-recorded by the Browser SDK are kept by default. --> | ||
|
|
||
| ## Creating a retention filter | ||
|
|
||
|
|
@@ -109,6 +109,45 @@ For example, to exclude sessions from South Korea while retaining all other sess | |
|
|
||
| **Note**: There is no way to prevent a specific event from being retained. You can use negative queries (for instance, adding `-@error.message:"Script error."` to a retention filter targeting RUM Errors) to minimize the volume of undesired events, but other retention filters may still make a positive retention decision about a session that contains the event you tried to filter out. | ||
|
|
||
| ## Cross-product retention filters | ||
|
|
||
| {{< callout url= | ||
| btn_hidden="false" header="Join the Preview!">}} | ||
| Cross-Product Retention Filters are in Preview. Use this form to submit your request today. | ||
| {{< /callout >}} | ||
|
|
||
| When configuring a RUM retention filter, you can enable two cross-product retention filters: one for session replays and one for APM traces. | ||
|
|
||
| - **Session Replay filter**: Retains replays for the specified percentage of sessions retained by the parent RUM retention filter which have an available replay. | ||
| - **APM traces filter**: Indexes APM traces for the specified percentage of sessions retained by the parent RUM retention filter which have available traces. | ||
| <div class="alert alert-info">The APM traces filter is only compatible with the following versions of the SDKs (applies to RUM and APM, but not Session Replay): <br> - Browser 6.5.0+ <br> - Android 3.0.0+ <br> - iOS 3.3.0+ <br> - React Native 3.0.0+ <br></div> | ||
|
|
||
| <div class="alert alert-danger">Configuring cross-product retention filters may increase APM-indexed volumes.</div> | ||
|
|
||
rtrieu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| **Note**: The availability of a session replay or APM traces depends on the initialization parameters `sessionReplaySampleRate` and `traceSampleRate` of the SDK. | ||
|
|
||
| The cross-product retention filters allow you to optimize the correlation between different products to retain richer telemetry. | ||
rtrieu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Example | ||
rtrieu marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| Consider a RUM retention filter configured as follows: | ||
|
|
||
| - **RUM retention filter**: 60% - `@type:error` (retention filter retaining 60% of sessions with at least one error) | ||
| - **Cross-product retention filter Session Replay**: 50% | ||
| - **Cross-product retention filter APM traces**: 25% | ||
|
|
||
| If across the sessions, 30% of them have an available replay and 40% of them have available traces, then the outcome is the following: | ||
|
|
||
| - 60% of sessions with at least one error are retained | ||
| - 50% × 30% = 15% of these retained sessions have a replay | ||
| - 25% × 40% = 10% of these retained sessions have APM traces | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This Example would be clearer with a screenshot and a more detailed explanation, I will re do it and give you another starting point that is clearer for customers |
||
| ### Default 1% retention | ||
|
|
||
| For compatible SDKs (see above), Datadog provides a default RUM retention filter and cross-product retention filter on APM traces that retains 1% of the sessions with available traces and their traces, at no additional cost. | ||
|
|
||
| This default filter ensures that you always have a baseline of correlated APM data available for your RUM sessions, even before custom cross-product retention filters. | ||
|
|
||
| ## Best practices | ||
|
|
||
| See [Retention Filter Best Practices][4]. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.