Skip to content

feat(mcp): Add per-tool-call metrics to logs episode response#24389

Merged
pelikhan merged 4 commits intomainfrom
copilot/add-per-tool-call-metrics
Apr 3, 2026
Merged

feat(mcp): Add per-tool-call metrics to logs episode response#24389
pelikhan merged 4 commits intomainfrom
copilot/add-per-tool-call-metrics

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 3, 2026

The logs response only exposed aggregate metrics per episode (total_tokens, mcp_failure_count), making it impossible to answer per-tool questions like which tool consumed the most tokens, which calls failed, or what the latency distribution looks like.

Changes

  • New EpisodeToolCall struct in logs_episode.go — lightweight per-call record with tool, server, tokens (estimated from InputSize+OutputSize / CharsPerToken), duration_ms (numeric, converted from formatted string), status, and error
  • ToolCalls []EpisodeToolCall added to EpisodeData, omitted from JSON when empty
  • mcpToolCallToEpisodeToolCall helper maps MCPToolCallEpisodeToolCall
  • buildEpisodeData updated to collect MCPToolCall records from each ProcessedRun.MCPToolUsage.ToolCalls and aggregate them across all runs in the episode

Each episode in the logs response now includes:

{
  "episode_id": "dispatch:wc-42",
  "total_tokens": 12840,
  "mcp_failure_count": 1,
  "tool_calls": [
    { "tool": "get_file_contents", "server": "github", "tokens": 2400, "duration_ms": 350, "status": "success" },
    { "tool": "search_code",       "server": "github", "tokens": 5200, "duration_ms": 1200, "status": "success" },
    { "tool": "create_pull_request", "server": "github", "tokens": 800, "duration_ms": 600, "status": "error", "error": "403 Resource not accessible by integration" }
  ]
}

For multi-run episodes (dispatch chains), tool calls are aggregated across all constituent runs.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw 9xL6IaqNl ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuTest User /usr/bin/git efaultBranchFromgit efaultBranchFromrev-parse ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name /opt/hostedtoolcache/node/24.14.1/x64/bin/npm GOINSECURE GOMOD GOMODCACHE npm itbr�� runs/20260403-222655-57611/test-715083458/.github/workflows GOPROXY /usr/local/bin/sh l GOWORK 64/bin/go sh (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo faul�� 2655-57611/test-2569216451 GO111MODULE /home/REDACTED/work/gh-aw/gh-aw/actions/setup/node_modules/.bin/sh GOINSECURE GOMOD GOMODCACHE sh -c ithub-script/git/ref/tags/v8 GOPROXY /bin/sh GOSUMDB GOWORK 64/bin/go /bin/sh (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name go1.25.8 -c=4 -nolocalimports -importcfg /tmp/go-build1981303199/b396/importcfg -pack /tmp/go-build1981303199/b396/_testmain.go env */*.ts' '**/*.js-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name --write ../../../**/*.jsGOMOD 64/bin/go --ignore-path ../../../.pretti/home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/.bin/pre�� /usr/bin/git go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name 0 -j ACCEPT GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.json' --ignore-path ../../../.prettierignore GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linusecurity (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --git-dir x_amd64/compile /usr/bin/git ck '**/*.cjs' '*git 1.4.1/auth/auth.rev-parse x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json GO111MODULE 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha /repos/actions/github-script/git/ref/tags/v8 l /usr/bin/git go1.25.8 -c=4 -nolocalimports git rev-�� --show-toplevel /home/REDACTED/go/pkg/mod/github.com/segmentio/asm@v1.1.3/keyset/keyset.go /usr/bin/git -json GO111MODULE ache/go/1.25.8/x--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha g_.a h00yucQ7c 64/pkg/tool/linux_amd64/link tierignore 1734482/b414/imprun 64/bin/go 64/pkg/tool/linu12345 env util.test GO111MODULE ortcfg.link GOINSECURE GOMOD GOMODCACHE H9I4Se4U3_YuAm2U5W/4Pkl_4aCjqpjptest@example.com (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --get-regexp ^remote\..*\.gh-resolved$ /usr/bin/git LsRemoteWithRealgit LsRemoteWithRealrev-parse 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/compile /usr/bin/git 1257870037/.githgit SUy_HbpQE 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel go /usr/bin/git /v3.0.0 GO111MODULE 64/pkg/tool/linu--show-toplevel git rev-�� .*/\1/p 64/pkg/tool/linux_amd64/vet /usr/bin/git t0 pkg/mod/github.crev-parse (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel epo}/actions/runs/3/artifacts /usr/bin/git e GO111MODULE x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git ub/workflows ii/equal_fold.gorev-parse x_amd64/compile git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel resolved$ /usr/bin/git Gitmain_branch22git Gitmain_branch22rev-parse x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git ub/workflows GO111MODULE x_amd64/compile git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile /usr/bin/git 5661736/b111/_pkgit GO111MODULE 1/x64/bin/node git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile /usr/bin/git vaScript23827607git GO111MODULE e/git git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha go1.25.8 -c=4 -nolocalimports -importcfg /tmp/go-build1125661736/b216/importcfg -pack /home/REDACTED/go/pkg/mod/github.com/segmentio/asm@v1.1.3/keyset/keyset.go env -json GO111MODULE ache/go/1.25.8/x-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha '**/*.ts' '**/*.-p GO111MODULE 1/x64/lib/node_m-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go 8d51�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha '**/*.ts' '**/*.-p cpgodebug/mcpgodebug.go 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/cgo GOINSECURE GOMOD GOMODCACHE x_amd64/cgo (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha -c go /usr/bin/git -json GO111MODULE x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git ub/workflows age/common.go x_amd64/compile git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha --show-toplevel epo}/actions/runs/4/artifacts /usr/bin/git e d x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git -json GO111MODULE x_amd64/compile git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha /tmp/go-build1125661736/b129/_pkg_.a -trimpath ache/node/24.14.1/x64/bin/node -p github.com/goccyrev-parse -lang=go1.21 /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile t-21�� sistency_GoAndJavaScript2382760713/001/test-simple-frontmatter.md -trimpath /usr/bin/git -p github.com/githurev-parse -lang=go1.25 git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha ets.TOKEN }} /tmp/go-build1981303199/b115/vet.cfg /usr/bin/git -json GO111MODULE x_amd64/compile git rev-�� --show-toplevel x_amd64/compile /usr/bin/git ub/workflows 1.4.1/internal/mrev-parse x_amd64/compile git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha 1303199/b387/embedcfg l /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -p crypto/sha512 -lang=go1.25 /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -o /tmp/go-build1981303199/b385/_pkg_.a -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -p main -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha v8qb/EvHVfoXKqPDWup4Av8qb -trimpath /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -p crypto/internal/rev-parse -lang=go1.25 /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile 1303�� /tmp/go-build1981303199/b384/_pkg_.a 1303199/b428/_testmain.go 1/x64/bin/node -p github.com/githurev-parse -lang=go1.25 1/x64/bin/node (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com /opt�� g_.a @v1.1.3/keyset/keyset.go 64/pkg/tool/linux_amd64/compile **/*.ts **/*.json --ignore-path 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 LsRemoteWithRealGitmain_branch2233663503/001' 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c g_.a 0/internal/catmsg/catmsg.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK abis 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name fer.go 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/asm -c npx prettier --check '**/*.cjs' '**/*.ts' '**/*.-p GOPROXY 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK 64/bin/go 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 5W/4Pkl_4aCjqpjptest@example.com -c g_.a /cpu/byteorder.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK run-script/lib/node-gyp-bin/sh 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c g_.a GOPROXY 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK 64/bin/go 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c 3245428224/.github/workflows oding@v0.5.4/iso8601/parse.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK 64/bin/go 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile /opt�� g_.a @v1.1.3/ascii/ascii.go 64/pkg/tool/linux_amd64/compile **/*.ts **/*.json --ignore-path 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c y_only_defaults_repo4031156580/001 4/apic.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK abis 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile /opt�� 3374616152/.github/workflows oding@v0.5.4/ascii/equal_fold.go 64/pkg/tool/linux_amd64/compile **/*.ts **/*.json --ignore-path 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c g_.a verutil.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK abis 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env hub/workflows GO111MODULE ache/go/1.25.8/x64/bin/go Action pins syngit GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name 3d4f6fb7c26b968ad 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile /opt�� g_.a --check 64/pkg/tool/linux_amd64/compile **/*.ts **/*.json --ignore-path 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c g_.a rotocol/go-sdk@v1.4.1/jsonrpc/jsonrpc.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK abis 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env hub/workflows GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE D7/90LsJdncQeTVLiE8t3kR/-pSRkXIpremote.origin.url /opt�� 3374616152/.github/workflows oding@v0.5.4/json/codec.go 64/pkg/tool/linux_amd64/compile **/*.ts **/*.json --ignore-path 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/pkg/tool/linux_amd64/asm GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/asm -c 2807429484/.github/workflows /semver/semver.go 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK 64/bin/go 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path 233663503/001' 233663503/001' de_modules/.bin/-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.js-p sonrpc2/conn.go 64/bin/go GOINSECURE GOMOD (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE 64/pkg/tool/linurev-parse env mLsRemoteWithRea@{u} mLsRemoteWithRealGitbranch_with_hyphen1438011429/001' x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build168346222/b001/cli.test /tmp/go-build168346222/b001/cli.test -test.testlogfile=/tmp/go-build168346222/b001/testlog.txt -test.paniconexit0 -test.timeout=3m0s -test.run=^Test -test.short=true GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE tions/setup/node_modules/.bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /tmp/go-build3454794867/b001/cli.test /tmp/go-build3454794867/b001/cli.test -test.testlogfile=/tmp/go-build3454794867/b001/testlog.txt -test.paniconexit0 -test.timeout=3m0s -test.run=^Test -test.short=true GO111MODULE /opt/hostedtoolcache/node/24.14.1/x64/bin/npx l GOMOD GOMODCACHE npx pret�� k/gh-aw/gh-aw/.github/workflows scripts/**/*.js /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/asm .prettierignore GOWORK 64/bin/go /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linurev-parse (http block)
    • Triggering command: /tmp/go-build4068658056/b001/cli.test /tmp/go-build4068658056/b001/cli.test -test.testlogfile=/tmp/go-build4068658056/b001/testlog.txt -test.paniconexit0 -test.timeout=3m0s -test.run=^Test -test.short=true LsRemoteWithRealGitmaster_branch1838014306/001' /usr/bin/git -json GO111MODULE x_amd64/compile /usr/bin/git remo�� -v (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --show-toplevel x_amd64/link /usr/bin/git g_.a lint:cjs 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel HD/45mdmEzBnpu2Sremote.origin.url /usr/bin/git g_.a UHEwN-QbQ sole.test git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha LsRemoteWithRealGitbranch_with_hyphen1438011429/001' LsRemoteWithRealGitbranch_with_hyphen1438011429/001' 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuTest User -c g_.a eUCGDzm1t 64/pkg/tool/linux_amd64/compile GOSUMDB GOWORK 64/bin/go 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha .js' --ignore-pa-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha h ../../../.pret.prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha .js' --ignore-path .prettierignore GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha .js' --ignore-pa-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/asm GOINSECURE GOMOD GOMODCACHE x_amd64/asm (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha .js' --ignore-pago1.25.8 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env cli/install.sh..." GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha .js' --ignore-pago1.25.8 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha h ../../../.prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha .js' --ignore-pa.artifacts[].name GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env t3433117049/.git--exclude-hidden=receive GO111MODULE /opt/hostedtoolc--quiet on; \ echo "���git GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha g_.a GO111MODULE 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile -c g_.a GOPROXY 64/pkg/tool/linux_amd64/vet GOSUMDB GOWORK 64/bin/go 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile /hom�� 28565954/001 **/*.cjs x_amd64/link **/*.json (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 1/x64/bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE /opt/hostedtoolcache/node/24.14.1/x64/bin/node GOINSECURE GOMOD GOMODCACHE node (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo -importcfg /tmp/go-build1981303199/b413/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil_test.go env */*.ts' '**/*.js-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo tions/setup/js/n-I GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.js-p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go --ignore-path ../../../.pretti/home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/.bin/pre�� /usr/bin/git go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build1981303199/b396/cli.test /tmp/go-build1981303199/b396/cli.test -test.testlogfile=/tmp/go-build1981303199/b396/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true -nolocalimports -importcfg /tmp/go-build1125661736/b189/importcfg -pack env -json GO111MODULE tions/node_modul-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /tmp/go-build163296214/b001/cli.test /tmp/go-build163296214/b001/cli.test -test.testlogfile=/tmp/go-build163296214/b001/testlog.txt -test.paniconexit0 -test.timeout=5m0s -test.run=^Test echo "��� FormatGOINSECURE git 64/bin/go tierignore ache/go/1.25.8/xenv /usr/bin/git go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /tmp/go-build168346222/b001/cli.test /tmp/go-build168346222/b001/cli.test -test.testlogfile=/tmp/go-build168346222/b001/testlog.txt -test.paniconexit0 -test.timeout=3m0s -test.run=^Test -test.short=true GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE tions/setup/node_modules/.bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE ache/go/1.25.8/x-lang=go1.25 GOINSECURE GOMOD GOMODCACHE go 1/x6�� -json GO111MODULE 64/pkg/tool/linu-lang=go1.25 GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linu-buildtags (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name --write ../../../**/*.jsGOMOD 64/bin/go --ignore-path ../../../.pretti-c /usr/bin/git go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name 0 -j ACCEPT GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.json' --ignore-path ../../../.prettierignore GO111MODULE 64/pkg/tool/linux_amd64/cgo GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linusecurity (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen1314447735/001' '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen1314447735/001' t GOINSECURE GOMOD GOMODCACHE sh -c "prettier" --check '**/*.cjs' '**/*.ts' '**/*.json' --ignore-path ../../../.prettierignore GOPROXY /opt/hostedtoolcache/go/1.25.8/x64/bin/go GOSUMDB GOWORK 64/bin/go go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch --show-toplevel go /usr/bin/infocmp repo717806205/00git GO111MODULE ules/.bin/prettier infocmp -1 xterm-color go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch licyBlockedUsersExpressionCompiledOutput2751355401/001 go /usr/bin/git GOPATH); \ if cogit GO111MODULE e/git git -C /tmp/TestCollectWorkflowFiles_TransitiveImports313495130/001 rev-parse /usr/bin/git 3 GO111MODULE /home/REDACTED/wor--get-regexp git (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Add `tool_calls` array to each episode in the `logs` response, exposing
per-call observability for token consumption, latency, and error details.

- Add `EpisodeToolCall` struct with tool, server, tokens, duration_ms, status, error
- Add `ToolCalls []EpisodeToolCall` field to `EpisodeData`
- Update `episodeAccumulator` to collect tool calls from processed runs
- Add `mcpToolCallToEpisodeToolCall` helper for MCPToolCall → EpisodeToolCall mapping
- Add `pkg/cli/logs_episode_test.go` with 4 tests for the new feature

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/487a4a21-144a-4e2b-8451-c5acb5f4c64d

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Add per-tool-call metrics to logs response feat(mcp): Add per-tool-call metrics to logs episode response Apr 3, 2026
Copilot AI requested a review from pelikhan April 3, 2026 22:31
@pelikhan pelikhan marked this pull request as ready for review April 3, 2026 22:35
Copilot AI review requested due to automatic review settings April 3, 2026 22:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds per-MCP-tool-call metrics to each episode in the logs response so consumers can analyze token usage, latency, and failures at a tool-call level (including across multi-run dispatch chains).

Changes:

  • Introduces EpisodeToolCall and adds tool_calls to EpisodeData rollups.
  • Aggregates per-run MCPToolUsage.ToolCalls into per-episode tool_calls via a conversion helper.
  • Adds unit tests covering inclusion/absence and cross-run aggregation of tool calls.
Show a summary per file
File Description
pkg/cli/logs_episode.go Adds episode-level tool_calls and mapping from MCPToolCallEpisodeToolCall during episode aggregation.
pkg/cli/logs_episode_test.go New tests validating tool-call inclusion, aggregation, and mapping behavior.
pkg/agentdrain/data/default_weights.json JSON formatting changes (single-line arrays) with no apparent semantic changes.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 2

Comment on lines +33 to +38
Tool string `json:"tool"`
Server string `json:"server"`
Tokens int `json:"tokens"`
DurationMS float64 `json:"duration_ms"`
Status string `json:"status"`
Error string `json:"error,omitempty"`
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duration_ms is represented as float64 here, but elsewhere in the CLI data models duration_ms is consistently an integer millisecond count (e.g., pkg/cli/token_usage.go uses DurationMs int json:"duration_ms"``). Using float64 for an integer millisecond value can cause unnecessary JSON float encoding and inconsistency for consumers. Consider changing `DurationMS` to `int`/`int64` (optionally with `omitempty` so missing/unknown durations don’t serialize as 0).

Suggested change
Tool string `json:"tool"`
Server string `json:"server"`
Tokens int `json:"tokens"`
DurationMS float64 `json:"duration_ms"`
Status string `json:"status"`
Error string `json:"error,omitempty"`
Tool string `json:"tool"`
Server string `json:"server"`
Tokens int `json:"tokens"`
DurationMS int64 `json:"duration_ms"`
Status string `json:"status"`
Error string `json:"error,omitempty"`

Copilot uses AI. Check for mistakes.
Comment on lines +189 to +193
// Collect per-tool-call metrics for this run.
if pr.MCPToolUsage != nil {
for _, tc := range pr.MCPToolUsage.ToolCalls {
acc.toolCalls = append(acc.toolCalls, mcpToolCallToEpisodeToolCall(tc))
}
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EpisodeData is described as a deterministic rollup and the code sorts RunIDs, WorkflowNames, episodes, and edges, but the new tool_calls slice is appended in the incoming runs iteration order and is never sorted. If the upstream processedRuns/runs ordering changes between invocations, tool_calls ordering will be nondeterministic and can create noisy diffs for the same underlying data. Consider sorting acc.toolCalls before assigning it (e.g., by timestamp if you include it, or otherwise by stable fields like server/tool/status/duration/tokens).

Copilot uses AI. Check for mistakes.
@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Apr 3, 2026

@copilot Merge main and recompile.

- Merge origin/main (adds Repository/Organization to EpisodeData)
- Change DurationMS from float64 to int64 for consistency with other
  duration_ms fields in the codebase
- Sort tool_calls deterministically by server, tool name, status
- Recompile all 183 workflow lock files

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/4ac0d3e9-652a-4288-ac51-bc36baf7d46d

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@copilot Merge main and recompile.

Done in commit 3d19726. Merged main (which adds Repository/Organization fields to EpisodeData), recompiled all 183 workflow lock files, and also applied the review feedback: changed DurationMS from float64 to int64 for consistency, and added deterministic sorting of tool_calls by server → tool name → status.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw %H %ct %D (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE ; \ fi -c &#34;prettier&#34; --cheGOSUMDB GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc-f (http block)
  • https://api.github.com/repos/github/gh-aw
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .visibility -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE r: $owner, name: $name) { hasDiscussionsEnabled } } GOINSECURE GOMOD GOMODCACHE x_amd64/link (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha npx prettier --cGOINSECURE GOPROXY $name) { hasDiscussionsEnabled } } GOSUMDB GOWORK 64/bin/go node /hom�� --check scripts/**/*.js 64/bin/go -d **/*.cjs 64/bin/go go (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Apr 3, 2026

@copilot merge make recompile

…call-metrics

# Conflicts:
#	pkg/cli/logs_episode.go

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@copilot merge make recompile

Done in commit 94d881f. Merged latest main and recompiled all 183 workflow lock files.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw -pack /home/REDACTED/work/gh-aw/gh-aw/cmd/gh-aw/main.go (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq .object.sha h ../../../.pret.prettierignore (http block)
  • https://api.github.com/repos/github/gh-aw
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .visibility on&#39; --ignore-pat-f (http block)
  • https://api.github.com/repos/githubnext/agentics/git/ref/tags/
    • Triggering command: /usr/bin/gh gh api /repos/githubnext/agentics/git/ref/tags/# --jq .object.sha w/js/**/*.json&#39; --ignore-path (http block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan pelikhan merged commit 2dfabd6 into main Apr 3, 2026
@pelikhan pelikhan deleted the copilot/add-per-tool-call-metrics branch April 3, 2026 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(mcp): Add per-tool-call metrics to logs response

3 participants