Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tail sampling processor #2572

Closed
kkondaka opened this issue Apr 21, 2023 · 3 comments · Fixed by #2497
Closed

Add tail sampling processor #2572

kkondaka opened this issue Apr 21, 2023 · 3 comments · Fixed by #2497
Assignees
Labels
enhancement New feature or request plugin - processor A plugin to manipulate data in the data prepper pipeline. untriaged
Milestone

Comments

@kkondaka
Copy link
Collaborator

Is your feature request related to a problem? Please describe.
Need a processor which does similar to what OTEL tail sampling processor does. More details can be found at - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/tailsamplingprocessor/README.md

Basic functionality is that - the decision to sample or not is taken after a trace is "complete". When it is complete if the trace is error trace, then it is allowed all the time (no sampling), if it is not an error trace then it is allowed based on configured percent using probabilistic sampling.
A trace is considered "complete" if there is no events/spans in the trace for a configured time period called "wait_period"

Describe the solution you'd like
Solution is support the behavior of tail sampling as supported by OTEL tail sampling processor. We use the existing aggregate processor framework in the Data Prepper to support this functionality. A new action with tail sampling functionality is the easy way to add this functionality to Data Prepper.

Describe alternatives you've considered (Optional)
I can't think of alternative approaches in the current Data Prepper. In future we could have better way to do tail sampling without using aggregate processor framework.

Additional context
Add any other context or screenshots about the feature request here.

@dlvenable dlvenable added this to the v2.3 milestone Apr 21, 2023
@dlvenable dlvenable added enhancement New feature or request plugin - processor A plugin to manipulate data in the data prepper pipeline. labels Apr 24, 2023
@dlvenable
Copy link
Member

@kkondaka , I'm re-opening this for you to address the requested comments in #2497.

@dlvenable dlvenable reopened this May 4, 2023
@kkondaka
Copy link
Collaborator Author

kkondaka commented May 4, 2023

@dlvenable I have already addressed comments in 2497 in a separate PR #2628

@dlvenable
Copy link
Member

Thanks!

@dlvenable dlvenable added this to 2.3 (May/Jun 2023) in Data Prepper Project Roadmap May 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request plugin - processor A plugin to manipulate data in the data prepper pipeline. untriaged
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

2 participants