Deduplicate tool-usage aggregation logic in audit report pipeline#23120
Merged
Deduplicate tool-usage aggregation logic in audit report pipeline#23120
Conversation
4 tasks
Replace the inline ~36-line tool-usage aggregation block in buildAuditData() (audit_report.go) with a call to the existing buildToolUsageInfo() helper (audit_agentic_analysis.go). This eliminates the duplicated code, standardizes sort behavior (both paths now sort by CallCount desc, then Name asc), and removes the now-unused workflow import from audit_report.go. Add regression tests to validate aggregation parity: - TestBuildToolUsageInfoAggregatesAndSorts: merging + max-merge rules - TestBuildToolUsageInfoEmpty: empty input - TestBuildToolUsageInfoSortOrderTieBreak: alphabetical tie-breaking - TestBuildAuditDataToolUsageMatchesBuildToolUsageInfo: parity check Agent-Logs-Url: https://github.com/github/gh-aw/sessions/3eeaac01-775c-45c8-95a3-cea0cdb7aec1 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] Fix duplicate tool-usage aggregation logic in audit report pipeline
Deduplicate tool-usage aggregation logic in audit report pipeline
Mar 26, 2026
pelikhan
approved these changes
Mar 26, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Deduplicates tool-usage aggregation in the audit report pipeline by reusing the existing buildToolUsageInfo(metrics) path so both audit code paths apply the same merge semantics and sorting.
Changes:
- Replaced the inline
metrics.ToolCallsaggregation logic inbuildAuditData()with a call tobuildToolUsageInfo(metrics). - Removed the now-unused
workflowimport fromaudit_report.go. - Added regression tests for aggregation behavior, sort order, empty input, and parity between
buildAuditData()andbuildToolUsageInfo().
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pkg/cli/audit_report.go | Uses buildToolUsageInfo(metrics) from buildAuditData() to centralize aggregation + sorting and drops unused import. |
| pkg/cli/audit_agentic_analysis_test.go | Adds regression tests for tool-usage merge + sort behavior and parity between audit paths. |
Comments suppressed due to low confidence (1)
pkg/cli/audit_agentic_analysis_test.go:279
- This test only checks that
MaxDurationis non-empty after merging, but the behavior being regressed is that the merged duration should be the maximum across calls. It would be more robust to assert the exact expected formatted duration (e.g., the 2s entry winning over the 1s entry) so a regression to “first seen” or “last seen” wouldn’t pass.
assert.Equal(t, 150, result[0].MaxInputSize, "max input size should be max across merged entries")
assert.Equal(t, 200, result[0].MaxOutputSize, "max output size should be max across merged entries")
assert.NotEmpty(t, result[0].MaxDuration, "max duration should be set from the longest call")
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+260
to
+275
| metrics := LogMetrics{ | ||
| ToolCalls: []workflow.ToolCallInfo{ | ||
| {Name: "bash", CallCount: 3, MaxInputSize: 100, MaxOutputSize: 200, MaxDuration: 1 * time.Second}, | ||
| {Name: "github_issue_read", CallCount: 5, MaxInputSize: 50, MaxOutputSize: 300, MaxDuration: 500 * time.Millisecond}, | ||
| {Name: "bash", CallCount: 2, MaxInputSize: 150, MaxOutputSize: 180, MaxDuration: 2 * time.Second}, | ||
| }, | ||
| } | ||
|
|
||
| result := buildToolUsageInfo(metrics) | ||
|
|
||
| require.Len(t, result, 2, "duplicate tool names should be merged") | ||
|
|
||
| // bash has CallCount 3+2=5 (merged), github_issue_read has CallCount 5. | ||
| // Both have equal call counts, so the tie is broken alphabetically: "bash" < "github_issue_read". | ||
| assert.Equal(t, "bash", result[0].Name, "alphabetically first among equal call counts should be first") | ||
| assert.Equal(t, 5, result[0].CallCount, "bash call counts should be summed: 3+2=5") |
There was a problem hiding this comment.
In TestBuildToolUsageInfoAggregatesAndSorts, both tools end up with the same CallCount (5 after merging), so the test only exercises the name tie-break and doesn’t actually validate the primary sort rule (CallCount descending). Consider adjusting the input counts (or adding an additional assertion/test) so at least one tool has a strictly higher CallCount and you can assert it sorts first.
This issue also appears on line 276 of the same file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ~36-line block aggregating
metrics.ToolCallsinto[]ToolUsageInfo(max-merge rules for input/output/duration) was duplicated betweenbuildAuditData()andbuildToolUsageInfo(), with the two paths already diverging:buildAuditDataomitted the sort thatbuildToolUsageInfoapplies.Changes
audit_report.go: Replace the inline aggregation block inbuildAuditData()with a call tobuildToolUsageInfo(metrics). Remove now-unusedworkflowimport.audit_agentic_analysis_test.go: Add regression tests covering:buildAuditData()andbuildToolUsageInfo()outputBoth audit paths now sort consistently.
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE ache/node/24.14.--show-toplevel git rev-�� --show-toplevel go /usr/bin/git *.json' '!../../git GO111MODULE 64/pkg/tool/linu--show-toplevel git(http block)https://api.github.com/orgs/test-owner/actions/secrets/usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ath ../../../.pr**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1/usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/infocmp o GO111MODULE(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v3/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha prettier l ache/node/24.14.0/x64/bin/node !../../../pkg/wogit --ignore-path ../../../.pretti--show-toplevel git t-14�� bility_SameInputSameOutput3743168357/001/stability-test.md go /usr/bin/git ignore-path ../.git GO111MODULE 64/bin/go git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v5/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha */*.ts' '**/*.json' --ignore-path ../../../.prettierignore GO111MODULE n-dir/node GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go son GOMOD GOMODCACHE erignore(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha needs.build.outputs.version go /usr/bin/git -json GO111MODULE x_amd64/link git rev-�� --show-toplevel x_amd64/link /usr/bin/git -json GO111MODULE tions/setup/js/n--show-toplevel git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel go /usr/bin/git -json GO111MODULE k/node_modules/.--show-toplevel git rev-�� --show-toplevel sh /usr/bin/git te '../../../**/git GOPROXY(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v6/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha k/gh-aw/gh-aw/.github/workflows/auto-triage-issues.md go /usr/bin/git w/js/**/*.json' git GO111MODULE 64/bin/go git chec�� .github/workflows/test.md go /opt/hostedtoolcache/node/24.14.0/x64/bin/node -json GO111MODULE 64/bin/go node(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel resolved$ /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --git-dir go /usr/bin/git -json GO111MODULE 64/bin/go git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel ger.test /usr/bin/git -json /node_modules/flrev-parse cal/bin/node git rev-�� --show-toplevel lag_test.go /usr/bin/git 2813-37448/test-git description_testrev-parse n_test.go git(http block)https://api.github.com/repos/actions/github-script/git/ref/tags/v8/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE modules/@npmcli/run-script/lib/node-gyp-bin/sh GOINSECURE GOMOD GOMODCACHE go env ath ../../../.pr**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env re --log-level=e!../../../pkg/workflow/js/**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha ath ../../../.pr**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env re --log-level=error GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-go/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha /tmp/go-build3189421196/b449/_pkg_.a -trimpath /usr/bin/git -p github.com/githurev-parse -lang=go1.25 git -C /tmp/gh-aw-test-runs/20260326-132813-37448/test-745862690 status /usr/bin/git .github/workflowgit -nolocalimports -importcfg git(http block)https://api.github.com/repos/actions/setup-node/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha k/gh-aw/gh-aw/.github/workflows/blog-auditor.md /tmp/file-tracker-test172061905/test2.lock.yml /usr/bin/git w/js/**/*.json' git GO111MODULE 64/bin/go git rev-�� --git-dir go /usr/bin/git -json GO111MODULE 64/bin/go git(http block)https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha qMJPDT0us go 0/x64/bin/node -json GO111MODULE 64/bin/go sh -has�� ithub/workflows/ace-editor.md stmain.go ache/go/1.25.0/x64/pkg/tool/linux_amd64/link tierignore GO111MODULE 64/bin/go ache/go/1.25.0/x64/pkg/tool/linux_amd64/link(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha -bool -buildtags(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha 2813-37448/test-745862690 go /home/node_modules/.bin/node -json GO111MODULE 64/bin/go node /hom�� --write scripts/**/*.js 0/x64/bin/node .prettierignore --log-level=errorev-parse 64/bin/go node(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha runs/20260326-132813-37448/test-2977102758/.github/workflows go /home/REDACTED/work/node_modules/.bin/node l GO111MODULE run-script/lib/n--show-toplevel node /hom�� --write scripts/**/*.js 0/x64/bin/node .prettierignore --log-level=errorev-parse 64/bin/go ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts/usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts/usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 0/x64/bin/bash GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ules/.bin/node GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts/usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE de/node/bin/bash GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE bin/node GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts/usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/node GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts/usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts/usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 0/x64/bin/node GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts/usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link env -json GO111MODULE 0/x64/bin/npx GOINSECURE GOMOD GOMODCACHE HC/wPHmRHH07drGotDxh6_4/9rUbv3kNVNgnGPLEQds7(http block)https://api.github.com/repos/github/gh-aw/actions/workflows/usr/bin/gh gh workflow list --json name,state,path re GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env w/js/**/*.json' --ignore-path ../../../.prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE go env ty-test.md GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha ty-test.md GO111MODULE tions/node_modules/.bin/sh GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.json' --ignore-path ../../../.prettierignore GO111MODULE 0/x64/bin/sh GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha -json GO111MODULE(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env js/**/*.json' ---errorsas GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE odules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/sh GOINSECURE GOMOD GOMODCACHE go env js/**/*.json' ---p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE /sh GOINSECURE GOMOD GOMODCACHE go env js/**/*.json' ---p GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env js/**/*.json' ---errorsas GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999/usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE ules/.bin/sh GOINSECURE GOMOD GOMODCACHE go env ithout_min-integrity1190069384/001 GO111MODULE cal/bin/sh GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/nonexistent/repo/actions/runs/12345/usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env '**/*.ts' '**/*.json' --ignore-path ../../../.pr**/*.json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/actions/workflows/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD iles,SysoFiles,T"prettier" --write '../../../**/*.json' '!../../../pkg/workflow/js/**/*.json' --ignore-path ../../../.prettierignore go(http block)/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE LcnGsjP9NU5w env -json GO111MODULE(http block)https://api.github.com/repos/owner/repo/contents/file.md/tmp/go-build3189421196/b403/cli.test /tmp/go-build3189421196/b403/cli.test -test.testlogfile=/tmp/go-build3189421196/b403/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/test-owner/test-repo/actions/secrets/usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ath ../../../.pr**/*.json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)If you need me to access, download, or install something from one of these locations, you can either:
💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.