Trace level retention filters docs #28349
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Preview links (active after the
|
Contributor
|
Created a docs card for review. |
brett0000FF
requested changes
Mar 26, 2025
Contributor
brett0000FF
left a comment
brett0000FF
left a comment
There was a problem hiding this comment.
Thanks for adding this content! This is an awesome new feature. Please take a look at my suggestions and let me know what you think. I'll do a quick review again once you review my suggestions. 🚀
| [14]: /monitors/types/apm/?tab=traceanalytics | ||
| [15]: /synthetics/apm/ | ||
| [16]: /security/application_security/ | ||
| [17]: /dynamic_instrumentation/ |
Contributor
There was a problem hiding this comment.
Suggested change
| [17]: /dynamic_instrumentation/ | |
| [17]: /dynamic_instrumentation/ | |
| [18]: https://app.datadoghq.com/apm/traces/retention-filters |
This is connected to my suggestion on line 114.
| 5. Save the new filter. | ||
|
|
||
| Filters are retained in a serial order. If you have an upstream filter that retains spans with the `resource:POST /hello_world` tag, those spans do not show up in the **Edit** window of a downstream filter that searches for spans with the same tag because they have been retained by the upstream filter. | ||
| **Note**: Configuring a trace rate might increase significantly your indexed spans usage. For instance, if you configure a retention filter to index spans from `service:my-service`: |
Contributor
There was a problem hiding this comment.
Suggested change
| **Note**: Configuring a trace rate might increase significantly your indexed spans usage. For instance, if you configure a retention filter to index spans from `service:my-service`: | |
| <div class="alert alert-danger">Configuring a trace rate can significantly increase your indexed spans usage.</div> | |
| For example, if you configure a retention filter to index spans from `service:my-service`: |
I think this could be worth bringing extra attention to with a danger alert box.
Comment on lines
121
to
122
| - Configuring a span rate of `100%` will ensure that all spans matching `service:my-service` will get indexed. | ||
| - Configuring a trace rate of `50%` will ensure that all spans from all traces with a span from `service:my-service` will get indexed. Assuming traces have 100 spans in average and 5 spans from `service:my-service`, configuring a trace rate will index the remaining 95 spans of the trace, for the trace rate percentage being configured. |
Contributor
There was a problem hiding this comment.
Suggested change
| - Configuring a span rate of `100%` will ensure that all spans matching `service:my-service` will get indexed. | |
| - Configuring a trace rate of `50%` will ensure that all spans from all traces with a span from `service:my-service` will get indexed. Assuming traces have 100 spans in average and 5 spans from `service:my-service`, configuring a trace rate will index the remaining 95 spans of the trace, for the trace rate percentage being configured. | |
| - Configuring a span rate of `100%` ensures that all spans matching `service:my-service` are indexed. | |
| - Configuring a trace rate of `50%` ensures that all spans from all traces with a span from `service:my-service` are indexed. Assuming traces have 100 spans in average and 5 spans from `service:my-service`, configuring a trace rate indexes the remaining 95 spans of the trace, for the trace rate percentage being configured. |
Tense fixes flagged by Vale.
| By default, spans indexed by custom retention filters **and** the intelligent retention filter are included in the Trace Explorer [aggregated views][6] (timeseries, toplist, table), as well as in dashboards and notebook queries. | ||
|
|
||
| However, because the diversity-sampled set of data is **not uniformly sampled** (that is, not proportionally representative of the full traffic) and is biased towards errors and high latency traces, you can choose to exclude these spans from these views by adding `-retained_by:diversity_sampling` query parameter to the query. | ||
| Spans indexed by custom retention filters **without a trace rate** |
Contributor
There was a problem hiding this comment.
Suggested change
| Spans indexed by custom retention filters **without a trace rate** |
Is this needed? Seems like the thought was cut off.
Co-authored-by: Brett Blue <84536271+brett0000FF@users.noreply.github.com>
brett0000FF
approved these changes
Apr 2, 2025
|
|
||
|
|
||
| In addition to these, you can create any number of additional [custom tag-based retention filters](#create-your-own-retention-filter) for your services, to capture the data that matters the most to your business. | ||
| After spans have been ingested, some are kept for 15 days according to the retention filters are set up on your account: |
Contributor
There was a problem hiding this comment.
Suggested change
| After spans have been ingested, some are kept for 15 days according to the retention filters are set up on your account: | |
| After spans have been ingested, some are kept for 15 days according to the retention filters that are set up on your account: |
|
|
||
| In addition to these, you can create any number of additional [custom tag-based retention filters](#create-your-own-retention-filter) for your services, to capture the data that matters the most to your business. | ||
| After spans have been ingested, some are kept for 15 days according to the retention filters are set up on your account: | ||
| 1. The **[Intelligent Retention Filter](#datadog-intelligent-retention-filter)** retains spans for every environment, service, operation, and resource for different latency distributions. |
Contributor
There was a problem hiding this comment.
Suggested change
| 1. The **[Intelligent Retention Filter](#datadog-intelligent-retention-filter)** retains spans for every environment, service, operation, and resource for different latency distributions. | |
| 1. The **[Intelligent Retention Filter](#datadog-intelligent-retention-filter)** retains spans for every environment, service, operation, and resource across different latency distributions. |
Contributor
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
dd-devflow
bot
added
mergequeue-status: queued
mergequeue-status: in_progress
and removed
mergequeue-status: queued
labels
dd-mergequeue
bot
deleted the
antoine.dussault/trace_level_retention_filters
branch
dd-devflow
bot
added
mergequeue-status: done
and removed
mergequeue-status: in_progress
labels
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
Trace level retention filter docs
Merge instructions
Merge readiness:
For Datadog employees:
Merge queue is enabled in this repo. Your branch name MUST follow the
<name>/<description>convention and include the forward slash (/). Without this format, your pull request will not pass in CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
To have your PR automatically merged after it receives the required reviews, add the following PR comment:
Additional notes