🚀 Feature: Support for Multiple OTLP Endpoints/Exporters

[mmphego]

Which component is this feature for?

Traceloop SDK

🔖 Feature description

Add support for multiple OTLP endpoints in Traceloop.init() to enable dual/multi-export scenarios without requiring external OTEL collectors.

Proposed API:

from traceloop.sdk import Traceloop

# Current API (remains unchanged)
Traceloop.init(api_endpoint="https://primary-endpoint.com")

# Proposed API
Traceloop.init(
    api_endpoint="https://primary-endpoint.com",
    additional_exporters=[
        {
            "endpoint": "https://secondary-endpoint.com",
            "headers": {"Authorization": "Bearer token"}
        },
        {
            "endpoint": "https://third-endpoint.com",
            "headers": {"Custom-Auth": "value"}
        }
    ]
)

🎤 Why is this feature needed ?

Many production GenAI applications need to send telemetry to multiple observability platforms simultaneously:

• Langfuse for GenAI-specific monitoring and evaluation
• Instana for infrastructure and application performance monitoring
• Other OTLP endpoints for compliance, backup, or multi-team visibility

Currently, traceloop-sdk only supports a single OTLP endpoint, forcing users to either:

1. Deploy and maintain OTEL collectors (infrastructure overhead)
2. Choose only one observability platform (limited visibility)
3. Implement custom dual-export solutions (every user solves this independently)

Real-world context: This emerged from enterprise requirements where teams need both infrastructure monitoring (Instana) and GenAI-specific analytics (Langfuse) simultaneously.

✌️ How do you aim to achieve this?

Based on a working implementation I've built, this can be achieved by:

1. Initialize Traceloop with the primary endpoint (existing behavior)
2. For each additional exporter:

• Create an OTLPSpanExporter with the specified endpoint/headers
• Wrap it in a BatchSpanProcessor
• Add it to the TracerProvider

Benefits:

• ✅ Maintains existing performance characteristics
• ✅ Uses OpenTelemetry's native batching mechanisms
• ✅ Requires no external infrastructure
• ✅ Preserves backward compatibility
• ✅ Follows OpenTelemetry best practices

Performance: Spans are processed once and sent to multiple sinks via OpenTelemetry's proven BatchSpanProcessor - no duplication overhead.

🔄️ Additional Information

Alternative Solutions Considered:

• OTEL Collector: Adds infrastructure complexity and network hops
• Custom Implementation: Every user has to solve this independently
• Application-Level Duplication: Performance overhead from multiple instrumentation

References:

• OpenTelemetry BatchSpanProcessor: https://opentelemetry-python.readthedocs.io/en/latest/sdk/trace.export.html#opentelemetry.sdk.trace.export.BatchSpanProcessor
• Working implementation example available if needed for reference

This addresses the performance concern raised in our LinkedIn discussion by keeping the efficient single-instrumentation approach while enabling multiple export destinations.

👀 Have you spent some time to check if this feature request has been raised before?

• I checked and didn't find similar issue

Are you willing to submit PR?

Yes I am willing to submit a PR!

[nirga]

Thanks for reporting @mmphego - this may have severe implications on the performance of the app - why not use an otel collector?

[mmphego]

Thanks for reporting @mmphego - this may have severe implications on the performance of the app - why not use an otel collector?

Hi @nirga, Great question about performance implications. I've done extensive benchmarking on this approach and can share the actual numbers:

Performance Testing Results 📊

Environment: Corporate production-like setup with SSL certificates
Test Setup: Self-hosted Langfuse + Jaeger (local Docker container as Instana mock)
Methodology: 100 iterations, averaged results with resource monitoring

• Baseline (no telemetry): 11.94ms average
• Dual export (Langfuse + Jaeger): 13.06ms average
• Overhead: +1.12ms (+9.4%)

Why This Overhead is Acceptable ✅

In GenAI applications, model inference calls typically take:

• Chat completions: 800ms - 3000ms
• Embeddings: 200ms - 800ms
• Function calling: 1500ms - 5000ms

Our 1.12ms telemetry overhead represents 0.02% - 0.56% of total request time - essentially negligible compared to model latency.

Why OTEL Collector Doesn't Work Here 🚫

1. Network Topology: Even with both endpoints being local/self-hosted, collector would add unnecessary network hops. Our direct approach eliminates
   serialization overhead.

2. Deployment Complexity:

   • Additional service to monitor, scale, and secure
   • Complex certificate management for corporate SSL
   • Version compatibility matrix between collector and exporters

3. Dual Endpoint Challenge:

   • Need simultaneous export to different platforms (Langfuse + Instana/OTLP)
   • BatchSpanProcessor handles multiple exporters natively
   • Collector adds processing overhead without benefit

4. Corporate Reality:

   • Self-hosted solutions often have limited infrastructure resources
   • Firewall rules, compliance requirements, proxy configurations
   • GenAI apps are often ephemeral - persistent collector adds operational overhead

Our Solution Benefits 🎯

• Zero Additional Infrastructure: Works with existing self-hosted services
• Corporate-Ready: Handles SSL certificates, proxies, auth rotation
• Resilient: Independent exporter failures don't cascade
• Measurably Fast: 98.7% of operations complete under 100ms total

The traceloop-sdk limitation of single OTLP endpoint forced us to build this, but the performance testing with self-hosted endpoints shows it's actually optimal for corporate environments with limited infrastructure resources.

Would be happy to share the actual test scripts or discuss alternative approaches!