Skip to content

Emit agent output metrics on OTLP conclusion spans for all outcomes#27495

Merged
pelikhan merged 2 commits intomainfrom
copilot/add-gh-aw-output-item-count
Apr 21, 2026
Merged

Emit agent output metrics on OTLP conclusion spans for all outcomes#27495
pelikhan merged 2 commits intomainfrom
copilot/add-gh-aw-output-item-count

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 21, 2026

sendJobConclusionSpan only read agent_output.json on failure/timeouts, so successful runs lacked output-volume telemetry in OTLP spans. This prevented querying and alerting on “success but zero outputs” and correlating output volume with token usage.

  • Conclusion span enrichment (all outcomes)

    • send_otlp_span.cjs now reads /tmp/gh-aw/agent_output.json unconditionally in sendJobConclusionSpan.
    • Adds gh-aw.output.item_count for every run.
    • Adds gh-aw.output.item_types as a sorted, deduplicated comma-separated list when item types exist.
  • Propagation to dedicated agent sub-span

    • The same enriched attribute set is reused for the gh-aw.<job>.agent span, so output metrics are present on both conclusion and agent spans when emitted.
  • Targeted test updates

    • Updated success-path tests to assert agent_output.json is read and output metrics are emitted.
    • Extended agent-span test coverage to assert gh-aw.output.item_count appears on both conclusion and agent spans.
const agentOutput = readJSONIfExists("/tmp/gh-aw/agent_output.json") || {};
const outputItems = Array.isArray(agentOutput.items) ? agentOutput.items : [];

attributes.push(buildAttr("gh-aw.output.item_count", outputItems.length));

const itemTypes = [...new Set(outputItems
  .map(i => (i && typeof i.type === "string" ? i.type : ""))
  .filter(Boolean))]
  .sort();

if (itemTypes.length > 0) {
  attributes.push(buildAttr("gh-aw.output.item_types", itemTypes.join(",")));
}

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login &#43; &#34;/&#34; &#43; .name 64/pkg/tool/linux_amd64/vet GOINSECURE contextprotocol//tmp/js-hash-test-1570574705/test-hash.js GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env 438365205 aMu6/n6X7R7Av3bGkLZAPaMu6 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link -c ger.test EgAi/JW5fl0E13YyEocudEgAi ortcfg.link --format=%H:%ct GOWORK 64/bin/go 9Ym34G_sfJyM6o-I^remote\..*\.gh-resolved$ (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env 438365205 ahb4/lZep-2MiwczJtV1iahb4 .cfg GOINSECURE l/ascii GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-tests -c 5303330/b394/_pkg_.a -ZkR/Y5KUpR6ZrQZn8hJV-ZkR .cfg -n1 --format=format:config --end-of-options--get-regexp ache/go/1.25.8/x^remote\..*\.gh-resolved$ (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 3370ef759399fede-d GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv --paginate repos/{owner}/{repo}/actions/runs/3/artifacts /usr/bin/gh .artifacts[].namgit .cfg 64/pkg/tool/linu--show-toplevel gh repo�� view owner/repo /usr/bin/git 89916/b252/_pkg_node r73k/ZR15bOYtzO_/opt/hostedtoolcache/node/24.14.1/x64/bin/npm ache/go/1.25.8/xinstall git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv GOMODCACHE go /usr/bin/git -json GO111MODULE 64/bin/node git rev-�� --show-toplevel go /usr/bin/git y_with_explicit_node GO111MODULE ache/go/1.25.8/xinstall git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv ansitiveImports780848707/001 remote /usr/bin/git -json GO111MODULE x_amd64/vet git init�� reflectlite/asm.. x_amd64/vet ache/node/24.14.1/x64/bin/node -json GO111MODULE x_amd64/vet ache/node/24.14.1/x64/bin/node (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv s/test.md rev-parse /usr/bin/git 7905d0a3804511efgit GO111MODULE 64/bin/go git rev-�� --show-toplevel go /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json GO111MODULE 64/bin/go /opt/hostedtoolcache/node/24.14.1/x64/bin/node (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv se 5303330/b107/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p io -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -o 2701117790 -trimpath 1/x64/bin/node -p compress/gzip -lang=go1.25 /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linu-trimpath /usr/bin/git e-analyzer.md sRemoteWithRealG\n .cfg git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linurev-parse /usr/bin/git 89916/b169/imporgit -trimpath ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/vet /usr/bin/git /tmp/go-build117git -trimpath 1/x64/bin/node git rev-�� --show-toplevel git /usr/bin/git ithub/workflows/git -goversion 5303330/b439/vet--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260421-051656-33976/test-1133774189/.github/workflows config /usr/bin/git remote.origin.urgit l/errors/error.grev-parse x_amd64/compile git conf�� --get remote.origin.url om/org1/repo.git -json GO111MODULE x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv GOMODCACHE go /usr/bin/git -json GO111MODULE 64/bin/go git -C /tmp/gh-aw-test-runs/20260421-052225-58908/test-2829424235/.github/workflows rev-parse /usr/bin/git hyphen20375248/0git hyphen20375248/0rev-parse 64/bin/go git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel M2/2rd54d-9cdbu-SqAYBLY/t9jI7Mg6origin /usr/bin/git CompiledOutput41git -trimpath 5303330/b394/act-b git rev-�� --show-toplevel 5303330/b394/actionpins.test /usr/bin/git t0 /tmp/go-build117rev-parse 1/x64/bin/node git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git SameOutput128773git GO111MODULE .cfg git rev-�� --show-toplevel erignore /usr/bin/git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -c=4 -nolocalimports -importcfg /tmp/go-build1175303330/b436/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/semverutil/semverutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/semverutil/semverutil_test.go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json age.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/compile /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json .cfg 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node /tmp�� github.event.issue.number 64/pkg/tool/linux_amd64/vet /usr/bin/git tcfg .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv 35633328/001 x_amd64/compile /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --show-toplevel git /usr/bin/git hub/workflows GO111MODULE 1/x64/lib/node_m--show-toplevel git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv user.email test@example.com /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json .cfg 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node /tmp�� github.actor 2y/osIhF1DWDut7__g-6NHb/U0bwBiAPrev-parse /usr/bin/git y-test.md .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv 35633328/001 x_amd64/compile /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --show-toplevel resolved$ /usr/bin/git hub/workflows GO111MODULE ache/go/1.25.8/x--show-toplevel git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ry=1 --json 5303330/b443/_pkg_.a 172605019/001&#39; 172605019/001&#39; -importcfg git conf�� s/test.md s/4/artifacts /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -json GO111MODULE x_amd64/compile /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv -bool -buildtags /opt/hostedtoolcache/node/24.14.1/x64/bin/node -errorsas -ifaceassert -nilfunc node /tmp�� ons-test1132524497 -tests /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv --count st/suppress-warn-w $name) { hasDiscussionsEnabled } } README.md ion-test..token-init de_modules/.bin//tmp/go-build2737938836/b001/_pkg_.a st/dist/workers/forks.js show�� --verify 8475de66 ules/.bin/node token-test.txt (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv 8475de66..HEAD --stdout tions/node_modules/.bin/git README.md ion-test..token--c n-dir/git git show�� --verify 8475de66..HEAD 86_64/node token-test.txt ings.cjs modules/@npmcli/graphql git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv -- st/suppress-warnings.cjs $name) { hasDiscussionsEnabled } } user.email test@test.com etch-fail st/dist/workers/forks.js k/gh�� --verify --quiet 1/x64/bin/node -u origin ules/.bin/git 1/x64/bin/node (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linuremote.origin.url /opt/hostedtoolcache/node/24.14.1/x64/bin/node XMdlm1J_w .cfg 64/pkg/tool/linu--show-toplevel node /tmp�� /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/artifacts-summary.md 64/pkg/tool/linux_amd64/vet /usr/bin/git ned-imports-enabgit LsRemoteWithRealrev-parse 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git conf�� --get remote.origin.url /usr/bin/git -json GO111MODULE 64/bin/node git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260421-051656-33976/test-279332281/.githu--detach rev-parse /usr/bin/git 289905082/001&#39; 289905082/001&#39; x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git -json /common.go x_amd64/compile git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/.github/workflows/architecture-guardian.md -goversion ache/node/24.14.1/x64/bin/node -c=4 -nolocalimports -importcfg ache/node/24.14.1/x64/bin/node s-19�� ets.TOKEN }} rev-parse /usr/bin/infocmp -json GO111MODULE modules/@npmcli/--get infocmp (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv 5303330/b455/timeutil.test -extld=gcc 5303330/b455/importcfg.link -json k x_amd64/compile R_3VA1bELNvSY/sg3YSHTzqdvJXL5t-fhz/8-8vmLiYCmHH9yLNKNaz/ITCHFh6R_3VA1bELNvSY -C ry=1 config ache/node/24.14.1/x64/bin/node remote.origin.urgit g/catalog.go x_amd64/compile git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv /tmp/TestHashConsistency_GoAndJavaScript1209511902/001/test-simple-frontmatter.md -goversion ache/node/24.14.1/x64/bin/node -c=4 -nolocalimports -importcfg ache/node/24.14.config s-23�� b.actor }}, Reporemote.origin.url go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name rg/x/text@v0.36.0/internal/tag/t-c=4 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD 89916/b013/symab--show-toplevel 64/pkg/tool/linux_amd64/vet env 3074592664/.github/workflows GO111MODULE .cfg GOINSECURE t/internal/strinrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/nistec/frev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env 89916/b246/_pkg_.a mW0N/BDDpIqgj5QBgNtEbmW0N ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/message/catalorev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-importcfg (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile env 152221/b403/_pkg_.a GO111MODULE 1/x64/bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE randutil GOMODCACHE 64/pkg/tool/linux_amd64/vet env 3483058789 SZyr/UNQkpBpW_IvLZuHOSZyr ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 om/modelcontextprotocol/go-sdk@v1.5.0/internal/mcpgodebug/mcpgodebug.go 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com env 438365205 gNV_/-ERQMY_tDmUJytyNgNV_ .cfg GOINSECURE contextprotocol/rev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linurev-parse (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE d8220e5f01623c1f2c05f44353774f0b8a703d98f05a05a86058133580d90b37-d GOINSECURE GOMOD ode-gyp-bin/sh go m/_n�� CommaSeparatedCompiledOutput121267004/001 GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 89916/b216/_pkg_.a GO111MODULE .cfg GOINSECURE g/x/text/secure/rev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-buildtags (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 om/modelcontextprotocol/go-sdk@v1.5.0/internal/j-ifaceassert 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/check GOMODCACHE 64/pkg/tool/linux_amd64/vet env 438365205 wDwi/8TvZlM4P0nfuVfRvwDwi .cfg GOINSECURE contextprotocol/init GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go m/_n�� CommaSeparatedCompiledOutput121267004/001 GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE hlite 89916/b007/symabis 64/pkg/tool/linux_amd64/vet ache�� 3074592664/.github/workflows _zAe/m6K4S-499xrKjIdi_zAe ache/go/1.25.8/x64/pkg/tool/linu-buildmode=exe GOINSECURE b/gh-aw/pkg/giturev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-extld=gcc (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/drbg GOMODCACHE 64/pkg/tool/linux_amd64/vet env 1258918842/custom/workflows NNuM/NZNs7zEf3uyY_7BzNNuM 64/pkg/tool/linux_amd64/compile GOINSECURE ce GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile env 152221/b395/_pkg_.a GO111MODULE 152221/b395=&gt; GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com env 89916/b252/_pkg_.a r73k/ZR15bOYtzO_sNGC5r73k ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE b/gh-aw/pkg/semvrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 89916/b021/atomirev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env 89916/b235/_pkg_.a 4ACQ/f02Eva1ttQPQuPWq4ACQ ache/go/1.25.8/x64/pkg/tool/linu-nilfunc GOINSECURE osh-tekuri/jsons/tmp/js-hash-test-27505538/test-hash.js GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-tests (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile env 2225-58908/test-2159411691 GO111MODULE 152221/b405=&gt; GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD 89916/b013/symabuser.name pBvTgXO/G1KutSxXTest User env 3074592664 89916/b013/importcfg ck GOINSECURE /semver GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-importcfg (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE entropy GOMODCACHE 64/pkg/tool/linux_amd64/vet env 89916/b236/_pkg_.a h5RJ/fhSiz4P0ozPJ9_2Hh5RJ 64/pkg/tool/linux_amd64/vet GOINSECURE osh-tekuri/jsonsrev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile env 2225-58908/test-2159411691 GO111MODULE 152221/b408=&gt; GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 3074592664/.github/workflows 7Ps3/Xuna8G_bMUX3GMM57Ps3 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/langurev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-buildtags (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 89916/b237/_pkg_.a DUdE/2oEXO76xEThYfB4YDUdE k GOINSECURE t/message GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile env 2225-58908/test-2159411691 GO111MODULE 152221/b410=&gt; GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path 172605019/001&#39; 172605019/001&#39; -importcfg /tmp/go-build1175303330/b418/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/tar.go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env from .github/aw to pkg/actionpins/data/action_pi-p GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 89916/b078/ GOMODCACHE 64/pkg/tool/linux_amd64/vet env 89916/b200/_pkg_.a .cfg .cfg GOINSECURE GOMOD 89916/b078/symab--get Vgol9MA/jtMHmSR1remote.origin.url (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build1175303330/b403/cli.test /tmp/go-build1175303330/b403/cli.test -test.testlogfile=/tmp/go-build1175303330/b403/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /tmp/go-build4235382655/b403/cli.test /tmp/go-build4235382655/b403/cli.test -test.testlogfile=/tmp/go-build4235382655/b403/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv it/ref/tags/v4 Eu/7mNRYRFA9RXlMxf9qL--/YcBrNqCW-buildtags sv EklR7_A7a -trimpath ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ortcfg /usr/bin/git g/fileutil/fileuls g/fileutil/tar.g-lh 64/pkg/tool/linu/tmp/gh-aw/aw-feature-branch.patch git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git sRemoteWithRealGgit sRemoteWithRealGrev-parse cal/bin/sh /usr/bin/git remo�� -v rtcfg /usr/bin/git 1243926c601af34bls GO111MODULE ache/go/1.25.8/x/tmp/gh-aw/aw-feature-branch.patch git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv Ahfx/EOBK6ELuzyeu-LDGAhfx GO111MODULE ck GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath sRem�� se 5303330/b144/vet.cfg .cfg -I /tmp/go-build840rev-parse -I ache/go/1.25.8/x64/pkg/tool/linumyorg (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go itma�� -json GO111MODULE bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv go GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env .a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv bytealg/indexbyte_wasm.s GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env Gitmain_branch11-errorsas Gitmain_branch11-ifaceassert x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env .a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE P7KsbSroa4S9 env tmatter-with-env-template-expressions.md GO111MODULE 2d54587cb92abf9e-d GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv se 5303330/b007/vet.cfg 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go eWit�� -json GO111MODULE bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE b/gh-aw/pkg/loggrev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go t-ha�� ithub/workflows/architecture-guardian.md GO111MODULE 1/x64/bin/node GOINSECURE GOMOD GOMODCACHE 1/x64/bin/node (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD cpu/cpu.s x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state .cfg GOINSECURE hpke GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile -Oz --enable-bu/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOMOD GOMODCACHE M7eNvRe/86NY3VUw-buildtags env -json 2/compile.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name with-tools.md GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 8c46274245fca23c-d GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 89916/b237/_pkg_.a DUdE/2oEXO76xEThYfB4YDUdE k GOINSECURE t/message GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet sRem�� Hyq-/iXwo-doyexxITIJ8Hyq- -trimpath .cfg -p internal/msan -lang=go1.25 ache/go/1.25.8/x4 (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch xterm-color go /opt/hostedtoolcache/node/24.14.1/x64/bin/node h ../../../.pretgit GO111MODULE 64/bin/go node /tmp�� /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/api-consumption-report.md go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git git conf�� --local --get ode_modules/.bin/git cal/bin/git git /git git add . git tions/setup/node_modules/.bin/git -M main bin/git git (dns block)
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git git form�� origin/auth-cleanup-success..auth-cleanup-success --stdout ode_modules/.bin/git -1 --format=%s tions/setup/nodefor-each-ref git add . git tions/setup/node_modules/.bin/git -m Initial commit p/bin/git git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Add gh-aw.output.item_count to conclusion spans for successful runs Emit agent output metrics on OTLP conclusion spans for all outcomes Apr 21, 2026
Copilot AI requested a review from pelikhan April 21, 2026 05:27
@pelikhan pelikhan marked this pull request as ready for review April 21, 2026 05:28
Copilot AI review requested due to automatic review settings April 21, 2026 05:28
@github-actions github-actions Bot mentioned this pull request Apr 21, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Ensures OTLP spans always include agent output-volume telemetry by reading agent_output.json for all job outcomes and emitting output metrics on both conclusion spans and the dedicated agent sub-span.

Changes:

  • Read /tmp/gh-aw/agent_output.json unconditionally in sendJobConclusionSpan.
  • Emit gh-aw.output.item_count for every run and gh-aw.output.item_types when available.
  • Update tests to assert agent_output.json is read on success and metrics appear on both conclusion and agent spans.
Show a summary per file
File Description
actions/setup/js/send_otlp_span.cjs Always reads agent output and adds output metrics to span attributes reused by both conclusion and agent spans.
actions/setup/js/send_otlp_span.test.cjs Updates/extends tests to validate unconditional reads and presence of output metrics on both spans.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 1

attributes.push(buildAttr("gh-aw.output.item_count", outputItems.length));
const itemTypes = [...new Set(outputItems.map(i => (i && typeof i.type === "string" ? i.type : "")).filter(Boolean))].sort();
if (itemTypes.length > 0) {
attributes.push(buildAttr("gh-aw.output.item_types", itemTypes.join(",")));
Copy link

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gh-aw.output.item_types is built from arbitrary agent_output.json content and can grow without bound (many unique/long types). While sanitizeOTLPPayload() will truncate strings for the HTTP export, the local JSONL mirror is written with the raw payload, so this new attribute can significantly inflate /tmp/gh-aw/otel.jsonl artifacts. Consider explicitly truncating the joined item_types string (and/or capping the number of types) before adding it to attributes, ideally reusing MAX_ATTR_VALUE_LENGTH.

Suggested change
attributes.push(buildAttr("gh-aw.output.item_types", itemTypes.join(",")));
const maxItemTypes =
typeof MAX_ATTR_VALUE_LENGTH === "number" && MAX_ATTR_VALUE_LENGTH > 0
? MAX_ATTR_VALUE_LENGTH
: 4096;
const boundedItemTypes = itemTypes.slice(0, maxItemTypes);
const joinedItemTypes = boundedItemTypes.join(",");
const truncatedItemTypes =
joinedItemTypes.length > maxItemTypes
? joinedItemTypes.slice(0, maxItemTypes)
: joinedItemTypes;
attributes.push(buildAttr("gh-aw.output.item_types", truncatedItemTypes));

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 80/100

Excellent test quality

Metric Value
New/modified tests analyzed 2
✅ Design tests (behavioral contracts) 2 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 2 (100%)
Duplicate test clusters 0
Test inflation detected Yes (24 test lines / 8 production lines = 3:1 ratio)
🚨 Coding-guideline violations None

Test Classification Details

View All Test Classifications
Test File Classification Notes
(agent span with agent output) — modified to assert gh-aw.output.item_count on both spans send_otlp_span.test.cjs:~1590 ✅ Design Verifies both agent span and conclusion span carry the output item count attribute
reads agent_output.json and adds output metrics when agent conclusion is success (renamed + expanded) send_otlp_span.test.cjs:2242 ✅ Design Verifies item_count and item_types attributes on the OTLP span for success conclusions

Test Analysis

✅ Modified: agent span attributes test (send_otlp_span.test.cjs:~1590)

Classification: Design test
What invariant is enforced? Both the agent span and the conclusion span must carry the gh-aw.output.item_count attribute propagated from agent_output.json.
Value if deleted? High — a regression that stripped output metrics from either span would go undetected.
Edge cases: The readFileSync mock throws ENOENT for all paths other than the expected file, verifying the code reads only the correct file.


✅ Modified: reads agent_output.json and adds output metrics when agent conclusion is success (send_otlp_span.test.cjs:2242)

Classification: Design test
What invariant is enforced? On a success conclusion, agent_output.json is read exactly once and its item metrics are emitted — item_count reflects the full item array length (including items with no type field), and item_types is sorted and deduplicated.
Value if deleted? High — would miss regressions in item count calculation and type deduplication logic.
Edge cases: Includes an item {} with no type field in the input array, verifying the count still includes it while the types string correctly omits its entry. Tests deduplication of repeated types ("pull_request" appears twice → "issue,pull_request").

Minor note: This test was previously asserting the opposite behavior ("does not read agent_output.json when agent conclusion is success"). The rename and inversion is accurate and the test now provides genuine behavioral coverage where there was none.


Score Breakdown

Component Points Earned Max
Behavioral Coverage (100% design tests) 40 40
Error/Edge Case Coverage (100% with edge cases) 30 30
Low Duplication (no duplicate clusters) 20 20
Proportional Growth (test inflation 3:1 > 2:1) 0 10
Total 80 100

i️ The 10-point deduction is for test inflation: 24 lines added to the test file vs. 8 lines in the production file (send_otlp_span.cjs). This is common when a feature's test harness requires more setup than the production change itself (mock configuration, span attribute assertions). The extra lines are justified here — they verify specific OTLP payload structure.


Language Support

Tests analyzed:

  • 🟨 JavaScript (*.test.cjs): 2 test cases — vitest, mocking fs.readFileSync (external I/O — acceptable)
  • 🐹 Go (*_test.go): 0 tests changed

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). Both modified tests verify observable behavioral contracts on OTLP span attributes. No coding-guideline violations detected.

📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.


References: §24705744118

🧪 Test quality analysis by Test Quality Sentinel · ● 544.8K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 80/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). Both modified tests enforce behavioral contracts on OTLP span attributes.

@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot review all comments. Max 64 item types

@pelikhan pelikhan merged commit 03a893d into main Apr 21, 2026
29 of 30 checks passed
@pelikhan pelikhan deleted the copilot/add-gh-aw-output-item-count branch April 21, 2026 05:41
Copilot stopped work on behalf of pelikhan due to an error April 21, 2026 05:42
github-actions Bot added a commit that referenced this pull request Apr 21, 2026
…trics

Documents OTLP conclusion span attributes for agent output metrics
introduced in PR #27495: agent_output.json is now read unconditionally
so error counts, output item counts, and token metrics are available
on all job outcomes (success, failure, timeout, cancellation).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[otel-advisor] add gh-aw.output.item_count to conclusion spans for successful runs

3 participants