Conversation
…t tools In the download goroutine in logs_orchestrator.go, extracted log metrics (Turns, TokenUsage, EstimatedCost) were stored in result.Metrics but not applied to result.Run before the fingerprint was computed. This meant processedRun.Run.Turns was 0 (stale from GitHub API) while audit.go correctly set run.Turns = metrics.Turns before computing the fingerprint. Fix: update result.Run fields from extracted metrics immediately after extraction, matching the pattern used in audit.go. Also compute Duration and ActionMinutes from timestamps at this point so all fields are consistent when deriveRunAgenticAnalysis is called. Adds TestDeriveRunAgenticAnalysisFingerprintConsistency to document and guard against this regression. Agent-Logs-Url: https://github.com/github/gh-aw/sessions/db696457-7e7c-4acd-b137-a1839c34f10a Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
logs and audit tools for the same run
There was a problem hiding this comment.
Pull request overview
Fixes a mismatch in behavior_fingerprint between the logs and audit tools by ensuring the logs download/concurrent processing path updates WorkflowRun fields from extracted log metrics before computing agentic analysis/fingerprints.
Changes:
- Sync
WorkflowRunfields (Turns,TokenUsage,EstimatedCost,LogsPath,Duration,ActionMinutes) from extracted log metrics/timestamps prior to fingerprint computation inlogs_orchestrator.go. - Add a regression unit test that demonstrates the fingerprint divergence when
Run.Turnsis stale vs. correctly populated.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| pkg/cli/logs_orchestrator.go | Updates the concurrent download path to populate result.Run from extracted metrics/timestamps before deriving agentic analysis and fingerprint. |
| pkg/cli/audit_agentic_analysis_test.go | Adds a regression test covering fingerprint differences caused by stale vs. synced Run.Turns. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Calculate duration and billable minutes from GitHub API timestamps. | ||
| // This mirrors the identical computation in audit.go so that | ||
| // processedRun.Run.Duration is consistent across both tools. | ||
| if !result.Run.StartedAt.IsZero() && !result.Run.UpdatedAt.IsZero() { | ||
| result.Run.Duration = result.Run.UpdatedAt.Sub(result.Run.StartedAt) | ||
| result.Run.ActionMinutes = math.Ceil(result.Run.Duration.Minutes()) | ||
| } |
There was a problem hiding this comment.
The comment says this duration + billable minutes computation “mirrors the identical computation in audit.go”, but audit.go only sets run.Duration (it does not set ActionMinutes). Please adjust the comment to match reality (or add the missing ActionMinutes calculation to audit if parity is intended) so future readers aren’t misled about cross-tool consistency.
| // Update run with metrics so fingerprint computation uses the same data | ||
| // as the audit tool, which also derives these fields from extracted log metrics. | ||
| result.Run.TokenUsage = metrics.TokenUsage | ||
| result.Run.EstimatedCost = metrics.EstimatedCost | ||
| result.Run.Turns = metrics.Turns | ||
| result.Run.LogsPath = runOutputDir | ||
|
|
There was a problem hiding this comment.
This “sync run fields from extracted metrics” mapping is now duplicated in several places (e.g., pkg/cli/audit.go:266-277, pkg/cli/audit_comparison.go:175-181, and here). Consider centralizing it into a helper to reduce the chance of future drift between tools (this bug was caused by a divergence like that).
The
logsandaudittools produced differentbehavior_fingerprintvalues for the same workflow run because the download goroutine inlogs_orchestrator.gocomputed the fingerprint before populatingresult.Runfrom extracted log metrics.Root cause
deriveRunAgenticAnalysisbuildsMetricsData.TurnsfromprocessedRun.Run.Turns, not from theLogMetricsargument. In the download goroutine,result.Metricswas set from parsed logs butresult.Run.Turns(andDuration,TokenUsage, etc.) remained at their zero/GitHub-API values.audit.gocorrectly doesrun.Turns = metrics.Turnsbefore buildingprocessedRun, so the two code paths diverged:logs(broken)audit(correct)Run.Turnsused for fingerprint0(stale from API)metrics.Turns(from log parse)execution_style"directed""exploratory"agentic_fraction00.5Fix
In the download goroutine, immediately after
result.Metrics = metrics, sync the same fields thataudit.gosets before computing the fingerprint:Test
TestDeriveRunAgenticAnalysisFingerprintConsistencydocuments the regression: it assertsexploratory/0.5+whenRun.Turnsis correctly set from metrics, anddirected/0.0when it is stale — showing exactly the divergence the fix closes.Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw 2662730/b247/vetrev-parse 86_64/bash git rev-�� --show-toplevel ache/go/1.25.0/x2 /usr/bin/git --noprofile(http block)https://api.github.com/orgs/test-owner/actions/secrets/usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name(http block)https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1/usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha user.name Test User /usr/bin/git get .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linuremote.origin.url /usr/bin/infocmp /workflows .cfg 64/pkg/tool/linu--show-toplevel infocmp(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v3/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha ithub/workflows/blog-auditor.md 957726/b418/_testmain.go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/link k/_temp/runtime-git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v5/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha ithub/workflows(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -v 64/pkg/tool/linux_amd64/vet /usr/bin/git --abbrev-ref HEAD h git rev-�� --show-toplevel ache/go/1.25.0/x64/pkg/tool/linurev-parse /usr/bin/git '**/*.ts' '**/*.git 2662730/b057/vetrev-parse de/node/bin/sh git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git -unreachable=falgit /tmp/go-build151rev-parse bin/node git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.0/x^remote\..*\.gh-resolved$ 0/x64/bin/node -bool on x_amd64/link git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v6/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha -test.paniconexit0 -test.v=true /opt/hostedtoolcache/node/24.14.0/x64/bin/node -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel node /tmp�� /home/REDACTED/work/gh-aw/gh-aw/.github/workflows/agent-persona-explorer.md 64/pkg/tool/linu-goversion /usr/bin/git get --local 64/pkg/tool/linu--show-toplevel git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha /tmp/TestHashStability_SameInputSameOutput2354393/001/stability-test.md x_amd64/vet /usr/bin/git js/**/*.json' --git --local 64/pkg/tool/linu--show-toplevel git conf�� --get remote.origin.url /usr/bin/git get .cfg odules/npm/node_--show-toplevel git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel /opt/hostedtoolcache/go/1.25.0/xtest@example.com /usr/bin/git -bool -buildtags cal/bin/node git rev-�� --show-toplevel erignore /usr/bin/git artifacts-summargit /tmp/go-build151rev-parse rgo/bin/bash git(http block)https://api.github.com/repos/actions/github-script/git/ref/tags/v8/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha ath ../../../.pr**/*.json(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build963957726/b435/importcfg -pack /tmp/go-build963957726/b435/_testmain.go conf�� w/js/**/*.json' --ignore-path --global 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/actions/setup-go/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha ub.actor }} x_amd64/vet /usr/bin/git js/**/*.json' --git ify@v1.11.1/requrev-parse 64/pkg/tool/linu--show-toplevel git -C /tmp/gh-aw-test-runs/20260329-104717-13773/test-1716342397 rev-parse /usr/bin/git @{u} .cfg 64/pkg/tool/linu--show-toplevel git(http block)https://api.github.com/repos/actions/setup-node/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha b.actor }}, Repo: ${{ github.repository }} x_amd64/vet /usr/bin/git js/**/*.json' --git --get 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linuconfig(http block)https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha ithub/workflows/agent-performance-analyzer.md --write /home/REDACTED/work/gh-aw/gh-aw/actions/setup/node_modules/.bin/sh **/*.ts **/*.json --ignore-path sh -c k/gh-aw/gh-aw/.github/workflows(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq .object.sha /tmp/TestCompileErrorFormatting1292243122/001 config /usr/bin/git remote.origin.urgit --get 64/pkg/tool/linu--show-toplevel git -C /tmp/gh-aw-test-runs/20260329-104717-13773/test-1716342397 rev-parse /usr/bin/git @{u} .cfg 64/pkg/tool/linu--show-toplevel git(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha "prettier" --write '**/*.cjs' '**/*.ts' '**/*.json' --ignore-patremote.origin.url -extld=gcc 957726/b395/vet.cfg(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha 4717-13773/test-1716342397 -tests(http block)https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts/usr/bin/gh gh run download 1 --dir test-logs/run-1 main x_amd64/compile(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts/usr/bin/gh gh run download 12345 --dir test-logs/run-12345 HEAD x_amd64/link(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts/usr/bin/gh gh run download 12346 --dir test-logs/run-12346 om/github/gh-aw .cfg(http block)https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts/usr/bin/gh gh run download 2 --dir test-logs/run-2 main tions/node_modules/.bin/node(http block)https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts/usr/bin/gh gh run download 3 --dir test-logs/run-3 main tions/setup/js/node_modules/.bin-buildmode=exe(http block)https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts/usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg x_amd64/link(http block)https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts/usr/bin/gh gh run download 5 --dir test-logs/run-5 main x_amd64/compile(http block)https://api.github.com/repos/github/gh-aw/actions/workflows/usr/bin/gh gh workflow list --json name,state,path(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 k/gh-aw/gh-aw/pkrev-parse k/gh-aw/gh-aw/pk--git-dir x_amd64/compile 0/x6�� se 2662730/b044/vet.cfg .cfg(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq .object.sha --show-toplevel x_amd64/compile /usr/bin/git se 2662730/b043/vetrev-parse ules/.bin/node git rev-�� --show-toplevel ache/go/1.25.0/xTest User /usr/bin/git se 2662730/b175/vets/-\{2,\}/-/g tions/setup/node_modules/.bin/sh--show-toplevel git(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha se 2662730/b085/vet.cfg .cfg(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha re --log-level=e!../../../pkg/workflow/js/**/*.json g/cli/logs_orche--ignore-path x_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha re --log-level=e!../../../pkg/workflow/js/**/*.json(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha re --log-level=error(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha re --log-level=e!../../../pkg/workflow/js/**/*.json(http block)https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999/usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha se 2662730/b090/vet-nolocalimports tions/setup/js/n-importcfg(http block)https://api.github.com/repos/nonexistent/repo/actions/runs/12345/usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion(http block)https://api.github.com/repos/owner/repo/actions/workflows/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/vet(http block)https://api.github.com/repos/owner/repo/contents/file.md/tmp/go-build963957726/b403/cli.test /tmp/go-build963957726/b403/cli.test -test.testlogfile=/tmp/go-build963957726/b403/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true r(http block)https://api.github.com/repos/test-owner/test-repo/actions/secrets/usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name(http block)If you need me to access, download, or install something from one of these locations, you can either:
📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.