Skip to content

Emit synthetic OTel exception events for failure conclusions without readable agent output#31334

Merged
pelikhan merged 3 commits into
mainfrom
copilot/grafana-otel-advisor-synthetic-exception
May 10, 2026
Merged

Emit synthetic OTel exception events for failure conclusions without readable agent output#31334
pelikhan merged 3 commits into
mainfrom
copilot/grafana-otel-advisor-synthetic-exception

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented May 10, 2026

✨ Enhancement

What does this improve?

Agent conclusion spans already marked failure as STATUS_CODE_ERROR, but when /tmp/gh-aw/agent_output.json was missing or unreadable they emitted no exception event. That left trace backends with a red span but no diagnosable failure signal to surface, group, or alert on.

Why is this valuable?

This keeps generic agent crashes aligned with the existing timed_out / cancelled synthetic-exception behavior, so failure traces remain queryable even when the agent never produces readable output.

  • Conclusion span behavior

    • Extend the synthetic-exception path in send_otlp_span.cjs to cover all non-OK agent conclusions with unreadable output, including failure.
    • Emit:
      • exception.type = gh-aw.AgentFailed
      • exception.message = agent failure
  • Failure-path coverage

    • Update the focused OTLP span tests to assert a synthetic exception event for:
      • missing agent_output.json
      • unreadable / invalid agent_output.json
    • Preserve the existing timeout and cancellation behavior unchanged.
  • Example

    if (hasNoReadableAgentOutput && isAgentNonOK) {
      let exceptionType = "gh-aw.AgentFailed";
      if (isAgentTimedOut) exceptionType = "gh-aw.AgentTimedOut";
      else if (isAgentCancelled) exceptionType = "gh-aw.AgentCancelled";
    }

Copilot AI and others added 2 commits May 10, 2026 12:28
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Emit synthetic exception event for agent failures Emit synthetic OTel exception events for failure conclusions without readable agent output May 10, 2026
Copilot AI requested a review from pelikhan May 10, 2026 12:30
@pelikhan pelikhan marked this pull request as ready for review May 10, 2026 12:33
Copilot AI review requested due to automatic review settings May 10, 2026 12:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves OTLP conclusion-span observability by emitting a synthetic OpenTelemetry exception event when the agent concludes non-OK (including failure) but /tmp/gh-aw/agent_output.json is missing or unreadable, ensuring trace backends have a queryable failure signal even without agent output.

Changes:

  • Extend synthetic-exception emission in sendJobConclusionSpan to trigger for any non-OK agent conclusion when there is no readable agent_output.json (adds gh-aw.AgentFailed for generic failures).
  • Add/adjust focused tests to assert synthetic exception events for missing and unreadable agent_output.json on failure.
Show a summary per file
File Description
actions/setup/js/send_otlp_span.cjs Emit synthetic exception events for all non-OK conclusions when agent output is absent/unreadable (including failure).
actions/setup/js/send_otlp_span.test.cjs Add assertions covering synthetic failure exceptions for missing/unreadable agent_output.json.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 0

@pelikhan pelikhan merged commit 7d83ae6 into main May 10, 2026
4 checks passed
@pelikhan pelikhan deleted the copilot/grafana-otel-advisor-synthetic-exception branch May 10, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[grafana-otel-advisor] OTel improvement: emit synthetic exception event for agent failures with missing output

3 participants