Skip to content

Python: fix: establish correct span parenting during streaming agent invocation#5098

Open
JasonOA888 wants to merge 1 commit intomicrosoft:mainfrom
JasonOA888:fix/5089-streaming-span-parenting
Open

Python: fix: establish correct span parenting during streaming agent invocation#5098
JasonOA888 wants to merge 1 commit intomicrosoft:mainfrom
JasonOA888:fix/5089-streaming-span-parenting

Conversation

@JasonOA888
Copy link
Copy Markdown

Problem

Closes #5089

When stream=True, the invoke_agent span is not activated as the current OTel context. Child spans (chat, execute_tool) are created without a parent reference and appear as siblings instead of children.

Expected:                      Actual (broken):
invoke_agent (WeatherAgent)    invoke_agent (WeatherAgent)
  ├── chat (gpt-5.4)           chat (gpt-5.4)              ← sibling
  ├── execute_tool             execute_tool                ← sibling
  └── chat (gpt-5.4)           chat (gpt-5.4)              ← sibling

Root Cause

The streaming path in _trace_agent_invocation uses bare get_tracer().start_span() without context activation. The original code deliberately avoided trace.use_span() because streaming spans are closed in cleanup hooks that run in a different async context, causing "Failed to detach context" errors.

Fix

Introduce _STREAMING_AGENT_INVOKE_CONTEXT — a contextvars.ContextVar that holds the OTel context with the agent-invoke span set as current.

Three changes:

  1. _trace_agent_invocation (streaming path): After creating the invoke_agent span, store trace.set_span_in_context(span) in the ContextVar. Reset in _close_span.

  2. _trace_chat_response (streaming path): Read the ContextVar and pass as context= to start_span(), so chat spans become children of invoke_agent.

  3. get_function_span: Read the ContextVar and pass as context= to start_as_current_span(), so execute_tool spans become children of invoke_agent.

This avoids use_span()/attach() entirely — no detach errors — while propagating parent context via Python contextvars.

Testing

  • All 14 existing streaming observability tests pass
  • All 90+ observability tests pass

Files Changed

  • python/packages/core/agent_framework/observability.py (+32 lines)

When stream=True, agent-invoke spans were not set as the active OTel
context, causing child spans (chat completion, execute_tool) to appear
as siblings rather than children of invoke_agent.

The original code avoided trace.use_span() for streaming spans due to
"Failed to detach context" errors when cleanup hooks run in a different
async context.

Fix: introduce _STREAMING_AGENT_INVOKE_CONTEXT ContextVar that holds
the OTel context with the agent-invoke span. Child spans read from this
to establish correct parent-child relationships without needing
use_span()/attach().

Closes microsoft#5089
@github-actions github-actions Bot changed the title fix: establish correct span parenting during streaming agent invocation Python: fix: establish correct span parenting during streaming agent invocation Apr 5, 2026
@moonbox3 moonbox3 requested a review from Copilot April 20, 2026 06:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes OpenTelemetry span hierarchy in Python streaming agent runs so chat and execute_tool spans correctly appear as children of the streaming invoke_agent span (closing issue #5089).

Changes:

  • Introduces a ContextVar (_STREAMING_AGENT_INVOKE_CONTEXT) to hold an OTel Context with the streaming invoke_agent span set as current.
  • Uses that stored context as the explicit parent when starting streaming chat spans.
  • Uses that stored context as the explicit parent when starting execute_tool spans via get_function_span().

Comment on lines +1569 to +1574
# Activate the agent-invoke span as the current OTel context so that
# child spans (chat completion, execute_tool) created during streaming
# inherit it as their parent.
agent_ctx = trace.set_span_in_context(span)
streaming_agent_ctx_token = _STREAMING_AGENT_INVOKE_CONTEXT.set(agent_ctx)

Copy link

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the streaming path, _STREAMING_AGENT_INVOKE_CONTEXT is set before any potential exceptions from _capture_messages(...)/attribute serialization. If an exception is raised here, the ContextVar token is never reset and the span is never ended, which can leak the streaming parent context into later operations in the same task. Consider wrapping the post-set(...) setup in a try/except/finally that resets the ContextVar (and ends the span) on early failures before the cleanup hooks are registered.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: Streaming mode breaks span parenting: chat/execute_tool spans not children of invoke_agent

3 participants