-
Notifications
You must be signed in to change notification settings - Fork 2.1k
Description
Issue: ADK trace exporters don't respect OpenTelemetry environment variables for IAP authentication
Problem Description
We are building agents with ADK using the get_fast_api_app
function and integrating with Langfuse for observability. While this works perfectly with a local Langfuse deployment, we encounter authentication issues when connecting to our production Langfuse instance deployed on GKE behind Google Identity-Aware Proxy (IAP).
Current Behavior
When ADK attempts to export traces to our IAP-protected Langfuse endpoint, we receive:
Failed to export span batch code: 401, reason: Invalid IAP credentials: empty token
This occurs despite setting the required OpenTelemetry environment variables for authentication headers.
Expected Behavior
ADK's trace exporters should respect the standard OpenTelemetry environment variables, particularly OTEL_EXPORTER_OTLP_HEADERS
, to include authentication headers when exporting traces.
Environment Variables Set
According to OpenTelemetry documentation, Langfuse's ADK integration guide and Google IAP programmatic authentication, we've configured:
OTEL_EXPORTER_OTLP_HEADERS="Proxy-Authorization=Bearer <valid-iap-token>"
OTEL_EXPORTER_OTLP_ENDPOINT="https://our-langfuse-instance.example.com"
Minimal code:
import os
from google.adk.cli.fast_api import get_fast_api_app
# Set OpenTelemetry headers for IAP authentication
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "Proxy-Authorization=Bearer <valid-token>"
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://langfuse-behind-iap.example.com"
# Create ADK FastAPI app with tracing enabled
app = get_fast_api_app(
agents_dir="/path/to/agents"
)
Analysis
The issue appears to be that ADK's internal trace exporter configuration doesn't inherit or respect the standard OpenTelemetry environment variables for headers. This prevents proper authentication with services behind IAP or other proxy authentication mechanisms.
Request
Could ADK be updated to properly respect the standard OpenTelemetry environment variables? Thank you.
Additional Context
ADK Version: 1.8.0
Python Version: 3.12
EDIT: Solved.