Skip to content

Add OTel exception span events for individual agent errors#25914

Merged
pelikhan merged 5 commits intomainfrom
copilot/otel-advisor-add-span-events-for-errors
Apr 12, 2026
Merged

Add OTel exception span events for individual agent errors#25914
pelikhan merged 5 commits intomainfrom
copilot/otel-advisor-add-span-events-for-errors

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 12, 2026

Agent errors are concatenated into a single gh-aw.error.messages attribute as a pipe-delimited string, making individual errors unqueryable in OTel backends. Backends like Grafana Tempo, Honeycomb, and Datadog have first-class support for the exception span event convention but can't use it today.

Changes

  • buildOTLPPayload: Accept optional events array, include in span object when non-empty
  • sendJobConclusionSpan: Emit one exception span event per error from agent_output.json, with exception.message truncated to MAX_ATTR_VALUE_LENGTH (1024)
  • Existing gh-aw.error.count and gh-aw.error.messages attributes are preserved (additive change)
  • Unlike the capped attribute (first 5 errors), span events emit all errors
// Each error becomes a structured, queryable span event
const spanEvents = outputErrors
  .map(e => e?.message ?? String(e))
  .filter(Boolean)
  .map(msg => ({
    timeUnixNano: errorTimeNano,
    name: "exception",
    attributes: [buildAttr("exception.message", msg.slice(0, 1024))],
  }));

Tests

10 new test cases covering: event inclusion/omission in buildOTLPPayload, per-error event emission, 1024-char truncation, success path (no events), absent file (no events), uncapped event count, valid timestamps.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw 9025215/b094/vetrev-parse g_.a git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git 1842483453 -trimpath 1/x64/bin/node git (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw remote.origin.urrev-parse (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.8/x64/bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --show-toplevel x_amd64/compile /opt/hostedtoolcache/node/24.14.1/x64/bin/node PefC8rlji .cfg 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node /tmp�� Value: ${{ github.actor }} 64/pkg/tool/linu-dwarf=false /usr/bin/infocmp 1251329/b094/_pkgit .cfg 64/pkg/tool/linu--show-toplevel infocmp (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha add myorg ache/node/24.14.1/x64/bin/node "prettier" --chegit sh x_amd64/vet ache/node/24.14.1/x64/bin/node 7550�� 64/bin/go x_amd64/vet /usr/bin/git --show-toplevel sh /usr/bin/git git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha /tmp/gh-aw-test-runs/20260412-154025-32333/test-1351631803/.gith@{u} rev-parse /usr/bin/git -json GO111MODULE x_amd64/compile git -C /tmp/gh-aw-test-runs/20260412-154025-32333/test-779531849/.github/workflows rev-parse om/owner/repo.git -json GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha k/gh-aw/gh-aw/.github/workflows/architecture-guardian.md rev-parse /usr/bin/git -json GO111MODULE ache/go/1.25.8/x--show-toplevel git -C /tmp/gh-aw-test-runs/20260412-154658-48773/test-3885631332 config /usr/bin/git remote.origin.urgit GO111MODULE tartedAt,updated--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha se 9025215/b036/vet.cfg .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/xrepos/{owner}/{repo}/actions/runs/12345/artifacts (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -test.paniconexit0 -test.v=true /usr/bin/git -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 3019955850/001 GO111MODULE ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel /opt/hostedtoolcache/go/1.25.8/x1 /usr/bin/git runs/20260412-15git -buildtags /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git ithub-script/gitgit -buildtags /usr/bin/git git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha --show-toplevel ache/go/1.25.8/x64/pkg/tool/linu-goversion /usr/bin/git se 9025215/b067/vet-f1 .cfg git rev-�� --show-toplevel ache/go/1.25.8/x12345 /usr/bin/git se 9025215/b105/vetrev-parse ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha --show-toplevel 64/pkg/tool/linux_amd64/compile /usr/bin/git g_.a go /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git gusK9_7Bt go /usr/bin/git /usr/bin/git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha -json 1.5.0/internal/mcpgodebug/mcpgodebug.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE jsonrpc2 GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha -json gset/set.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha image:v1.0.0 remote.origin.url /usr/bin/git g_.a GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git m/workflows sYAOo28ie 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha --show-toplevel go /usr/bin/git ub/workflows /test2.lock.yml /usr/local/sbin/--show-toplevel git conf�� --get remote.origin.url /usr/bin/git licyTrustedUsersgit GOPROXY o.git git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/ace-editor.md x_amd64/vet /usr/bin/git ub/workflows til.go x_amd64/vet git rev-�� --git-dir x_amd64/vet /usr/bin/git g_.a k1Ubnk-ff 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha --show-toplevel ache/go/1.25.8/xshow /usr/bin/git -json GO111MODULE /usr/local/.ghcu--show-toplevel git remo�� add upstream /usr/bin/git licyTrustedUsersgit GOPROXY x_amd64/compile git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha ithub/workflows/agentic-observability-kit.md -buildtags /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -o /tmp/go-build3369025215/b429/_pkg_.a -trimpath /usr/bin/git -p main -lang=go1.25 git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha /tmp/go-build3208386788/b393/_pkg_.a l /usr/lib/git-core/git -p main -lang=go1.25 /usr/lib/git-core/git --gi�� for-each-ref --format=%(objectname) /opt/hostedtoolcache/node/24.14.1/x64/bin/node go1.25.8 -c=4 -nolocalimports node (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha /001/noflag-a.md -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p crypto/internal/rev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -o /tmp/go-build3451251329/b230/_pkg_.a -trimpath 9025215/b192/vet.cfg -p github.com/modelrev-parse -lang=go1.25 qwUng2b/iYjk4gCZirxzyXVtYgkS (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha RequiresMinIntegrity3359090621/001 9025215/b254/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p github.com/githurev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -uns�� -unreachable=false /tmp/go-build3369025215/b016/vet.cfg /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet go1.25.8 -c=4 -nolocalimports /opt/hostedtoolcache/go/1.25.8/x--json (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha 4025-32333/test-918436132 -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -p vendor/golang.orrev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -uns�� 9025215/b415/gitutil.test /tmp/go-build3369025215/b175/vet.cfg 9025215/b415/importcfg.link -goversion go1.25.8 -c=4 1hGYX3-aQLWqt/l-fEmuZxTCue0RDK6Brepos/{owner}/{repo}/actions/runs/5/artifacts (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha /tmp/gh-aw-test-runs/20260412-154025-32333/test-1842483453 resolved$ /usr/bin/git @{u} _56Gjvce9 x_amd64/vet git conf�� user.name Test User /usr/bin/git b/workflows GO111MODULE 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha --get remote.origin.url /usr/bin/git ithub/workflows/git tname) /home/REDACTED/.lo--show-toplevel git rev-�� --show-toplevel sh /usr/bin/git 4345-46691/test-git GOPROXY /home/REDACTED/.do--show-toplevel git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha sistency_GoAndJavaScript1570569998/001/test-simple-frontmatter.md -trimpath ache/node/24.14.1/x64/bin/node -p main -lang=go1.25 /usr/lib/git-core/git t-60�� bility_SameInputSameOutput2553502327/001/stability-test.md --revs /usr/bin/git --thin --delta-base-offrev-parse -q git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha 4658-48773/test-1447778411 go 8386788/b439/_pkg_.a y_with_repos_arrgit GO111MODULE 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -ato�� :latest -buildtags /usr/bin/git -errorsas -ifaceassert -nilfunc git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha /tmp/go-build3369025215/b405/_pkg_.a -trimpath /bin/sh -p github.com/githurev-parse -lang=go1.25 /bin/sh -c :latest git-receive-pack '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen1700110053rev-parse /usr/bin/git -c=4 -nolocalimports -importcfg git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha 4658-48773/test-1447778411 go /usr/bin/git y_with_repos_arrgit GO111MODULE ache/go/1.25.8/x--show-toplevel /usr/bin/git remo�� -v go /opt/hostedtoolcache/node/24.14.--limit 259724355/customgit GO111MODULE ache/go/1.25.8/x--show-toplevel node (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 1251329/b213/_pkg_.a go .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE randutil GOMODCACHE 64/pkg/tool/linux_amd64/vet env ternal/tools/actions-build/main.go SZyr/UNQkpBpW_IvLZuHOSZyr .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env ithout_min-integrity2647479314/001 GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE lite GOMODCACHE 64/pkg/tool/linux_amd64/vet env 1251329/b192/_pkg_.a .cfg x_amd64/vet GOINSECURE GOMOD 1251329/b006/sym--show-toplevel x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go 1/x6�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/subtle GOMODCACHE 64/pkg/tool/linux_amd64/vet env 1251329/b190/_pkg_.a xxol/GL-tkTNtkvunLaxVxxol util.test GOINSECURE GOMOD GOMODCACHE util.test (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 1251329/b078/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env 1251329/b201/_pkg_.a GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE fips140/hkdf 1251329/b078/sym--git-dir 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name itcustom_branch2886177394/002/work 1/x64/lib/node_modules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/node GOINSECURE GOMOD GOMODCACHE go 1/x6�� y_with_explicit_repo1095410642/001 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com ortc�� 1251329/b195/_pkg_.a stmain.go 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 om/yosida95/urit-nolocalimports 64/pkg/tool/linu-importcfg GOINSECURE /execenv GOMODCACHE 64/pkg/tool/linu/home/REDACTED/work/gh-aw/gh-aw/scripts/lint_error_messages_test.go env 2431930992/.github/workflows vNkW/MmwpPo_3e3tB-Au8vNkW util.test GOINSECURE g/x/net/idna GOMODCACHE util.test (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name LsRemoteWithRealGitcustom_branch2886177394/001' ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env ithout_min-integrity2647479314/0remote.origin.url GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE 1251329/b078/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env 918436132 GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE g/x/crypto/chachrev-parse ache/go/1.25.8/x--git-dir 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/pkg/tool/linu-nolocalimports GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linu/tmp/go-build3369025215/b455/_testmain.go env ternal/tools/generate-action-metadata/main.go aMu6/n6X7R7Av3bGkLZAPaMu6 .cfg GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name itbranch_with_hyphen4222509442/002/work tions/setup/js/node_modules/.bin/sh GOINSECURE GOMOD GOMODCACHE go env ut2546736470/001 GO111MODULE 86_64/bash GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 918436132 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 om/modelcontextprotocol/go-sdk@v1.5.0/internal/x-ifaceassert 64/pkg/tool/linux_amd64/vet GOINSECURE /unix GOMODCACHE 64/pkg/tool/linutest@example.com env 2431930992/.github/workflows 7LFx/9x5EhNlMwHDxpQFH7LFx .cfg GOINSECURE a95/uritemplate/rev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GO111MODULE 1/x64/lib/node_modules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/sh GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.json' --ignore-path ../../../.prettierignore GO111MODULE n-dir/bash GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/hmac GOMODCACHE 64/pkg/tool/linux_amd64/vet env 2431930992 ahb4/lZep-2MiwczJtV1iahb4 .cfg GOINSECURE fips140/ecdh GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name LsRemoteWithRealGitbranch_with_hyphen4222509442/001' tions/setup/node_modules/.bin/sh GOINSECURE GOMOD GOMODCACHE go env ut2546736470/001 GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -c=4 -nolocalimports -importcfg /tmp/go-build3369025215/b411/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/tar.go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env -json .go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE 64/pkg/tool/linuremote2 env rity3359090621/001 .cfg x_amd64/compile GOINSECURE b/gh-aw/pkg/typeinit GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github%2Fworkflows%2Faudit-workflows.md
    • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node --experimental-import-meta-resolve --require /home/REDACTED/.npm/_npx/69c381f8ad94b576/node_modules/vitest/suppress-warnings.cjs --conditions node --conditions development /home/REDACTED/.npm/_npx/69c381f8ad94b576/node_modules/vitest/dist/workers/forks.js git n-dir/git git chec�� -b fallback-branch ules/.bin/node . git rgo/bin/git git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git edcfg wDwi/8TvZlM4P0nfrev-parse ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linurev-parse /usr/bin/git 9025215/b395/_pk/bin/sh flge/CEDVAjFSK2L-c x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --git-dir git /usr/bin/git assword=$GITHUB_git assword=$GITHUB_rev-parse /usr/bin/git git rev-�� --show-toplevel git /opt/hostedtoolcache/node/24.14.1/x64/bin/node efaultBranchFrom/bin/sh efaultBranchFrom-c /usr/bin/git /opt/hostedtoolcgit-upload-pack 'origin' (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha edOutput3944586524/001 .cfg 64/pkg/tool/linux_amd64/compile GOINSECURE b/gh-aw/pkg/loggrev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env ub/workflows /test2.lock.yml /usr/local/sbin/sh GOINSECURE GOMOD GOMODCACHE sh (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env /a.out GO111MODULE x_amd64/vet GOINSECURE jYsH4YQuI9baCZTU-C GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha re GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha RequiresMinIntegrity4221348958/001 GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD ode-gyp-bin/sh ache/go/1.25.8/x-test.timeout=10m0s env Gitmaster_branch4286615595/001' Gitmaster_branch4286615595/001' ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x-importcfg (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env 944116/001 944116/002/work x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a @v1.1.3/base64/base64.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go 1439�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD erignore ache/go/1.25.8/x-buildtags env 740745160/001 740745160/002/wo-ifaceassert ache/go/1.25.8/x-nilfunc GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x-tests (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 1251329/b133/ GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env ub/workflows resolved$ /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/.bin/node GOINSECURE GOMOD GOMODCACHE node (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE fips140/drbg ache/go/1.25.8/xuser.name 64/pkg/tool/linuTest User env 1251329/b238/_pkg_.a NNuM/NZNs7zEf3uyY_7BzNNuM x_amd64/link GOINSECURE t/feature/pluralinit GOMODCACHE x_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go tion�� -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion rity2647479314/0git GO111MODULE x_amd64/link 8386788/b419/semverutil.test t-ha�� ithub/workflows/api-consumption-report.md stmain.go /usr/bin/gh -json GO111MODULE 64/bin/go gh (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json gset/set.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ath ../../../.pr**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build3369025215/b397/cli.test /tmp/go-build3369025215/b397/cli.test -test.testlogfile=/tmp/go-build3369025215/b397/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /tmp/go-build2122650759/b397/cli.test /tmp/go-build2122650759/b397/cli.test -test.testlogfile=/tmp/go-build2122650759/b397/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true rror infocmp /usr/bin/git go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /tmp/go-build3208386788/b224/cli.test /tmp/go-build3208386788/b224/cli.test -test.testlogfile=/tmp/go-build3208386788/b224/testlog.txt -test.paniconexit0 -test.timeout=10m0s 1/x6�� y_with_repos=public_2344837820/0remote.origin.url GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json 4 x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git git rev-�� origin/auth-cleanup-success git ules/.bin/git main..emoji-suffgit --stdout e_modules/.bin/g-u git conf�� user.name Test User bin/git -1 --format=%s 1/x64/lib/node_magent-change.txt git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI and others added 2 commits April 12, 2026 15:31
Agent-Logs-Url: https://github.com/github/gh-aw/sessions/6bdc94c1-f53f-409c-b275-415885eaf9ea

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Following the OpenTelemetry exception event semantic convention, each
agent error is now emitted as a separate span event with name="exception"
and an "exception.message" attribute, making individual errors queryable
in backends like Grafana Tempo, Honeycomb, and Datadog.

Changes:
- Add `events` parameter to `buildOTLPPayload` in send_otlp_span.cjs
- Emit one exception span event per error in `sendJobConclusionSpan`
- Add tests for events support in buildOTLPPayload (4 tests)
- Add tests for exception events in sendJobConclusionSpan (6 tests)

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/6bdc94c1-f53f-409c-b275-415885eaf9ea

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile

Copilot AI changed the title [WIP] Add OTel span events for individual agent errors Add OTel exception span events for individual agent errors Apr 12, 2026
Copilot AI requested a review from pelikhan April 12, 2026 15:59
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 12, 2026

@copilot merge main and recompile

Merged main — branch was already up to date. Ran make build and make recompile (187/187 workflows compiled successfully) with no changes produced. All 167 tests pass.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha {commit} (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v7 --jq .object.sha to pkg/workflow/data/action_pins.json...&#34; (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha (http block)
  • https://api.github.com/repos/docker/build-push-action/git/ref/tags/v7
    • Triggering command: /usr/bin/gh gh api /repos/docker/build-push-action/git/ref/tags/v7 --jq .object.sha (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/-
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/- --jq .object.sha (http block)

If you need me to access, download, or install something from one of these locations, you can either:


validate_workflows:
if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.operation == 'validate' && !github.event.repository.fork }}
runs-on: ubuntu-latest
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot update compiler to use Ubuntu-latest for this job

@pelikhan pelikhan marked this pull request as ready for review April 12, 2026 16:21
Copilot AI review requested due to automatic review settings April 12, 2026 16:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves OpenTelemetry export for agent failures by emitting per-error exception span events (in addition to the existing aggregated gh-aw.error.* attributes), making individual errors queryable in common OTel backends.

Changes:

  • Extend buildOTLPPayload to accept an optional events array and include it in the exported span only when non-empty.
  • Update sendJobConclusionSpan to emit one exception span event per agent error (with exception.message truncated to 1024 chars).
  • Add/extend unit tests covering event inclusion/omission, per-error event emission, truncation, and success/absent-file behavior.
Show a summary per file
File Description
pkg/workflow/maintenance_workflow.go Updates condition-comments (but introduces smart-quote typos that should be corrected).
actions/setup/js/send_otlp_span.cjs Adds span event support to the OTLP payload and emits OTel exception events for agent errors.
actions/setup/js/send_otlp_span.test.cjs Adds tests for event behavior in buildOTLPPayload and sendJobConclusionSpan.
.github/workflows/agentics-maintenance.yml Switches runner label for validate_workflows and removes the explicit Docker daemon startup step (may break docker-backed validation tooling).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

pkg/workflow/maintenance_workflow.go:757

  • This comment uses a smart quote character (” ) instead of the empty string literal "". Update the condition example to use "" so it matches the code (BuildStringLiteral("")).
// buildRunOperationCondition creates the condition for the unified run_operation
// job that handles all dispatch operations except the ones with dedicated jobs.
// Condition: dispatch && operation != ” && operation != each excluded && !fork.
func buildRunOperationCondition(excludedOperations ...string) ConditionNode {
  • Files reviewed: 4/4 changed files
  • Comments generated: 3

Comment on lines 738 to 741
// buildNotForkAndScheduledOrOperation creates a condition for jobs that run on
// schedule (or empty operation) AND when a specific operation is selected.
// Condition: !fork && (not_dispatch || operation == '' || operation == op)
// Condition: !fork && (not_dispatch || operation == || operation == op)
func buildNotForkAndScheduledOrOperation(operation string) ConditionNode {
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition examples in this comment use a smart quote character (” ) where an empty string literal is intended. This makes the documentation misleading and can also cause copy/paste errors. Replace ” with "" (empty string) to match the actual condition logic used below.

This issue also appears on line 754 of the same file.

Copilot uses AI. Check for mistakes.
Comment on lines 144 to 149
startTimeUnixNano: toNanoString(startMs),
endTimeUnixNano: toNanoString(endMs),
status,
attributes,
...(events && events.length > 0 ? { events } : {}),
},
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

buildOTLPPayload now supports events, but sanitizeOTLPPayload() only sanitizes resource/span attributes and does not traverse span.events[].attributes. This means event attributes bypass redaction/truncation guarantees when exporting. Extend sanitizeOTLPPayload to sanitize event attributes as well (and add/adjust tests accordingly).

Copilot uses AI. Check for mistakes.
@@ -315,11 +315,6 @@
- name: Build gh-aw
run: make build

Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validate_workflows job runs gh aw compile --validate --zizmor --actionlint --poutine (via run_validate_workflows.cjs), and those scanners are executed via Docker (see e.g. pkg/cli/zizmor.go, pkg/cli/actionlint.go, pkg/cli/poutine.go). This PR removes the step that ensures the Docker daemon is running, which can cause validation to fail on runners where Docker isn’t started by default. Consider restoring the Docker start/check (e.g. docker info with a fallback to start the service) before running validation.

Suggested change
- name: Ensure Docker is running
run: |
docker info >/dev/null 2>&1 || {
sudo systemctl start docker || sudo service docker start
}
docker info

Copilot uses AI. Check for mistakes.
@github-actions github-actions bot mentioned this pull request Apr 12, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 84/100

Excellent test quality

Metric Value
New/modified tests analyzed 10
✅ Design tests (behavioral contracts) 10 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 8 (80%)
Duplicate test clusters 0
Test inflation detected Yes — 201 test lines / 20 production lines = 10:1 (threshold: 2:1)
🚨 Coding-guideline violations None

Test Classification Details

📋 All 10 test classifications
Test File Classification Issues Detected
"includes events array in span when events are provided" send_otlp_span.test.cjs ✅ Design
"includes multiple events when provided" send_otlp_span.test.cjs ✅ Design
"omits events from span when events array is empty" send_otlp_span.test.cjs ✅ Design Edge case: empty array boundary
"omits events from span when events is not provided" send_otlp_span.test.cjs ✅ Design Edge case: optional param omitted
"emits one exception span event per error on agent failure" send_otlp_span.test.cjs ✅ Design Failure branch coverage
"truncates exception.message to 1024 characters" send_otlp_span.test.cjs ✅ Design Edge case: boundary value (2000-char string)
"does not emit exception events when agent conclusion is success" send_otlp_span.test.cjs ✅ Design Success-branch coverage
"does not emit exception events when agent_output.json is absent on failure" send_otlp_span.test.cjs ✅ Design Error path: missing file gracefully handled
"emits exception events for all errors (not capped at 5 like error messages attribute)" send_otlp_span.test.cjs ✅ Design Boundary: explicitly tests ≥ 5-item behaviour
"sets valid timeUnixNano on each exception event" send_otlp_span.test.cjs ✅ Design Protocol compliance: timestamp format validated with regex

Flagged Tests — Requires Review

No tests were flagged for quality issues.

Note — test inflation (informational only): send_otlp_span.test.cjs added 201 lines while the production file send_otlp_span.cjs added 20 lines (10:1 ratio). This triggers the inflation-penalty rule (-10 pts) but is expected and appropriate here: each of the 6 integration-style tests in sendJobConclusionSpan requires substantial boilerplate to stub fetch, set environment variables, and navigate deeply-nested OTLP JSON. The tests are high-quality and the penalty is a mechanical artefact of the rubric rather than a genuine concern.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 0 tests — no Go test files changed
  • 🟨 JavaScript (*.test.cjs, *.test.js): 10 tests (vitest)

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). All 10 new tests verify observable, user-visible behavioral contracts: correct OTLP span structure, truncation at 1024 bytes, conditional event emission based on agent conclusion, and graceful handling of missing files. The feature is well-covered.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 459.9K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 84/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). All 10 new JavaScript tests verify observable behavioral contracts for the OTel exception span events feature.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments

@pelikhan pelikhan merged commit aa3a9e6 into main Apr 12, 2026
@pelikhan pelikhan deleted the copilot/otel-advisor-add-span-events-for-errors branch April 12, 2026 16:41
Copilot AI requested a review from pelikhan April 12, 2026 16:41
Copilot stopped work on behalf of pelikhan due to an error April 12, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[otel-advisor] add OTel span events for individual agent errors (follow exception semantic convention)

3 participants