Feature Request: Tracing Sampling Rate Configuration for Langfuse

[himansrivastava]

Describe the feature or potential improvement

Description

I would like to request the addition of a tracing sampling rate configuration option to Langfuse. This feature would allow users to set a specific sampling rate for tracing, enabling more efficient and manageable logging practices.

Benefits

1. Improved Performance: By allowing users to set a sampling rate, Langfuse can reduce the volume of traces collected, which can significantly improve application performance and reduce overhead. This is particularly beneficial in high-throughput environments where tracing every single request can be resource-intensive.

2. Cost Efficiency: Reducing the number of traces collected can lead to substantial cost savings, especially when using cloud-based service that charge based on data volume. Users can achieve better cost management without compromising on critical insights.

3. Enhanced Focus on Critical Data: Sampling allows users to focus on the most relevant traces, making it easier to identify and troubleshoot issues. This targeted approach can improve the efficiency of debugging and monitoring activities.

4. Scalability: As applications grow in complexity and user base, the ability to control the tracing rate becomes essential. This feature will support the scalability of Langfuse, ensuring it remains effective and manageable at larger scales.

5. User Flexibility: Providing a configurable sampling rate empowers users to customize their tracing strategy according to their specific needs and constraints. Different applications and environments have varying requirements, and this flexibility ensures broader applicability and user satisfaction.

Proposed Implementation

• Configuration Parameter: Introduce a new parameter in the environment variables to set the tracing sampling rate (e.g., TRACING_SAMPLING_RATE).
• Rate Value: The parameter should accept a value between 0 and 1, representing the percentage of requests to trace (e.g., a value of 0.1 would trace 10% of the requests).
• Random Sampling: Implement random sampling logic to select which requests to trace based on the specified rate.
• Documentation Update: Provide clear documentation and examples on how to configure and use the sampling rate feature.

I believe this feature will greatly enhance the utility and performance of Langfuse and look forward to its consideration.

Additional information

No response

[marcklingen]

Thanks for raising this, totally agree that this would be extremely useful.

Thoughts

• Sampling is the most meaningful if it happens on the client side (SDKs, integrations) and on the trace-level (traces are complete, x% of traces get dropped instead of being send to the API)

Option 1: Sampling with random number -> complex as it needs to be implemented one-by-one for the different integrations

• The current SDKs and integrations are mostly stateless for maximum client-side performance and memory efficiency. This raises the challenge as to where to keep track of which trace to actually send to the Langfuse API. Most likely there is no single approach that works across the different integrations.
  • Llamaindex/Langchain/Haystack: These integrations have their own internal traceIds and keep the trace ids in state anyway. We could add a flag to each trace to indicate whether to include it. All observations of this trace could then be dropped.
  • Low-level SDKs with stateful clients (trace.generation(), ...): we could keep the sampling state in the trace client object and drop all subsequent observations which depend on this trace
  • Low-level sdks with custom ids (distributed implementation)
    • We'd need to keep a list of ids in state in order to sample them

Option 2: Sampling by applying a function to the trace_id -> stateless

• For each of the events send to the API, the trace_id is available.
• We could apply a deterministic function to the trace_id to decide whether to include the id in the sample or not.

Example implementation

import random
def deterministic_sample(trace_id, sample_rate):
    random.seed(42)  # Fixed seed for reproducibility
    hash_value = hash(uuid)
    normalized_hash = (hash_value & 0xFFFFFFFF) / 2**32  # Normalize to [0, 1)
    return normalized_hash < sample_rate

I think the first implementation is difficult to maintain and might introduce a lot of memory leakage problems when run at scale. The second would be much easier to implement. What do you think?

[maxdeichmann]

@himansrivastava thanks for the thoughts. We implemented it in the Python SDK release. Check out the docs on how to use it.