Add tail sampling processor #2572
Labels
enhancement
New feature or request
plugin - processor
A plugin to manipulate data in the data prepper pipeline.
untriaged
Milestone
Is your feature request related to a problem? Please describe.
Need a processor which does similar to what OTEL tail sampling processor does. More details can be found at - https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/tailsamplingprocessor/README.md
Basic functionality is that - the decision to sample or not is taken after a trace is "complete". When it is complete if the trace is error trace, then it is allowed all the time (no sampling), if it is not an error trace then it is allowed based on configured percent using probabilistic sampling.
A trace is considered "complete" if there is no events/spans in the trace for a configured time period called "wait_period"
Describe the solution you'd like
Solution is support the behavior of tail sampling as supported by OTEL tail sampling processor. We use the existing aggregate processor framework in the Data Prepper to support this functionality. A new action with tail sampling functionality is the easy way to add this functionality to Data Prepper.
Describe alternatives you've considered (Optional)
I can't think of alternative approaches in the current Data Prepper. In future we could have better way to do tail sampling without using aggregate processor framework.
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: