fix(otel): stop sending traces to api.openai.com and cleanup - #2400
Conversation
There was a problem hiding this comment.
Pull request overview
This PR hardens and cleans up OpenTelemetry integration around the OpenAI Agents SDK runtime to prevent unintended trace export to OpenAI’s hardcoded ingest endpoint (which can leak non-OpenAI gateway keys and cause 401s), while preserving OpenTelemetry-based tracing.
Changes:
- Drop the OpenAI Agents SDK native trace processor by default (opt-in via
KAGENT_OPENAI_AGENTS_NATIVE_TRACING=true) and instrument the SDK viaOpenAIAgentsInstrumentor. - Ensure
Resource.create(...)is used soOTEL_RESOURCE_ATTRIBUTESandtelemetry.sdk.*attributes are honored. - Ensure Gemini instrumentation still runs when
OTEL_LOGGING_ENABLED=true, and remove a no-opexcluded_urlsargument fromHTTPXClientInstrumentor().instrument(...).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| python/packages/kagent-openai/tests/test_tracing.py | Adds regression tests proving the Agents SDK default exporter targets OpenAI, and that kagent drops it by default while keeping OTel spans working. |
| python/packages/kagent-openai/src/kagent/openai/_a2a.py | Introduces _configure_openai_agents_tracing() to remove native exporting (by default) and warn when SDK tracing is disabled, then wires it into app startup. |
| python/packages/kagent-core/tests/test_tracing_configure.py | Adds tests ensuring Google/Gemini instrumentation still runs in the logging-only branch and that OTEL_RESOURCE_ATTRIBUTES are merged into the Resource. |
| python/packages/kagent-core/src/kagent/core/tracing/_utils.py | Switches to Resource.create(...), removes ineffective excluded_urls for HTTPX instrumentor, and ensures Google/Gemini instrumentation also runs when logging is enabled. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
e90b6d4 to
7109dd5
Compare
dhaifley
left a comment
There was a problem hiding this comment.
Looks good, but what if we just used the newer version mentioned in the PR description? >=0.52.3,<0.53.0? I'm not familiar enough to know if there is a reason we cannot do that.
Also, does the Go ADK code need a similar fix: go/adk/pkg/telemetry/tracing.go:81-84?
|
Thanks @dhaifley, addressed your comments and fixed a new bug with |
c69c8ff to
52fdeac
Compare
…urce attrs Signed-off-by: krisztianfekete <git@krisztianfekete.org>
Signed-off-by: krisztianfekete <git@krisztianfekete.org>
52fdeac to
7cb25f1
Compare
The Agents SDK registers a trace processor that POSTs everything to a hardcoded
https://api.openai.com/v1/traces/ingestusingOPENAI_API_KEY. It ignoresOPENAI_API_BASE, so anyone behind other proxies is shipping their gateway's key to OpenAI and getting 401s back.OpenAIAgentsInstrumentoradds the OTel processor next to that one rather than replacing it, andset_tracing_disabled(True)would silence the OTel spans too, so we now drop the built-in processor before instrumenting. SetKAGENT_OPENAI_AGENTS_NATIVE_TRACING=trueto keep it if you're using a real OpenAI key.Using
OpenAIAgentsInstrumentor(replace_existing_processors=True)as suggested in review, with the pin bumped to>=0.52.3,<0.53.0since that's when the kwarg landed. Doing it through the kwarg also means a secondbuild()can't wipe the OTel processor, sinceinstrument()no-ops once instrumented.Cleaned up a couple of other smaller stuff:
Resource(...)->Resource.create(...): the bare constructor ignoresOTEL_RESOURCE_ATTRIBUTESand dropstelemetry.sdk.*, so nobody could setdeployment.environment.nameorservice.versionat all.resource.Newstarts empty so it needsWithFromEnv()+WithTelemetrySDK().HTTPXClientInstrumentor().instrument(excluded_urls=...)did nothing, cleaned it up._a2a.pyreadself.config.kagent_urlbutKAgentConfigonly hasurl, which blows up wheneverKAGENT_URLisn't set.Rebased onto main, which dropped the Gemini bullet from this PR since main landed the same fix independently.