Skip to content

feat(otel): emit gen_ai.response.finish_reasons on the agent span#31332

Merged
pelikhan merged 2 commits into
mainfrom
copilot/otel-improvement-add-finish-reasons
May 10, 2026
Merged

feat(otel): emit gen_ai.response.finish_reasons on the agent span#31332
pelikhan merged 2 commits into
mainfrom
copilot/otel-improvement-add-finish-reasons

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 10, 2026

✨ Enhancement

What does this improve?
readAgentRuntimeMetrics already parsed the agent's stdio result JSON line but discarded stop_reason. Without it, gen_ai.response.finish_reasons was never set on the gh-aw.<job>.agent span, making it impossible to distinguish successful completions from silent max_tokens truncations in any OTel backend.

Implementation approach:

  • buildArrayAttr helper — new function that produces a properly-encoded OTLP arrayValue attribute (required for gen_ai.response.finish_reasons which is string[] in the GenAI semconv)
  • readAgentRuntimeMetrics — extracts stop_reason from the result JSON alongside the existing num_turns / total_cost_usd; stored as stopReason on AgentRuntimeMetrics
  • sendJobConclusionSpan — pushes gen_ai.response.finish_reasons: [stopReason] onto agentAttributes when present; omitted when the field is absent (e.g. cancelled/timed-out runs)

The resulting agent span now enables backend queries like:

# Detect silently truncated runs
{ gen_ai.response.finish_reasons =~ "max_tokens" }

Docs: gen_ai.response.finish_reasons added to the agent span attributes table in frontmatter.md.

- Add `buildArrayAttr` helper for OTLP string-array attributes
- Extend `AgentRuntimeMetrics` typedef with `stopReason` field
- Extract `stop_reason` from agent stdio log result JSON
- Emit `gen_ai.response.finish_reasons` on the dedicated agent span
- Add three tests covering present/absent/max_tokens scenarios
- Document `gen_ai.response.finish_reasons` in frontmatter.md

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add gen_ai.response.finish_reasons to the agent span feat(otel): emit gen_ai.response.finish_reasons on the agent span May 10, 2026
Copilot AI requested a review from pelikhan May 10, 2026 12:23
@pelikhan pelikhan marked this pull request as ready for review May 10, 2026 12:33
Copilot AI review requested due to automatic review settings May 10, 2026 12:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for emitting gen_ai.response.finish_reasons on the dedicated agent span by extracting stop_reason from the agent stdio result line and encoding it as an OTLP arrayValue attribute, enabling observability backends to distinguish normal completions from truncations like max_tokens.

Changes:

  • Add buildArrayAttr() helper to encode string[] OTLP attributes via arrayValue.
  • Extend readAgentRuntimeMetrics() to capture stop_reason and attach it to the agent span as gen_ai.response.finish_reasons.
  • Update docs and add unit tests verifying the attribute is included/omitted appropriately.
Show a summary per file
File Description
docs/src/content/docs/reference/frontmatter.md Documents gen_ai.response.finish_reasons as an agent span attribute.
actions/setup/js/send_otlp_span.cjs Parses stop_reason and emits gen_ai.response.finish_reasons on the agent span using a new array-attribute helper.
actions/setup/js/send_otlp_span.test.cjs Adds coverage for inclusion/omission of gen_ai.response.finish_reasons based on presence of stop_reason.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 0

@pelikhan pelikhan merged commit 66caf85 into main May 10, 2026
4 checks passed
@pelikhan pelikhan deleted the copilot/otel-improvement-add-finish-reasons branch May 10, 2026 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[otel-advisor] OTel improvement: add gen_ai.response.finish_reasons to the agent span (currently dropping stop_reason)

3 participants