[ab-advisor] Add output_format A/B experiment to deep-report workflow by Copilot · Pull Request #30447 · github/gh-aw

Copilot · 2026-05-05T20:24:11Z

Implements the output_format experiment campaign on deep-report.md, testing whether a condensed report format reduces token cost while maintaining discussion engagement.

Frontmatter changes

Adds experiments: output_format block with three variants: full_briefing (34%), executive_brief (33%), annotated_brief (33%)
15 min samples per variant, Mann-Whitney U analysis, starts 2026-05-06
Guardrail metrics use threshold expression form ("==0", ">=0.8") as required by the schema
owner and issue: 0 omitted — former is not in schema, latter requires ≥1 (add issue number once created)

Prompt body changes

Replaces the static Report Structure section with variant-aware conditionals using native {{#elseif}} syntax (per @pelikhan):

{{#if experiments.output_format == "executive_brief"}}
Generate a **condensed intelligence brief** with these sections only:
1. **🔍 Executive Summary** — 3 sentences: overall health, top finding, urgent action.
2. **🚨 Top 5 Findings** — Flat bullet list, one line each, most impactful first.
3. **✅ Actionable Agentic Tasks** — Exactly 7 items as before.
{{#elseif experiments.output_format == "annotated_brief"}}
Generate a **condensed intelligence brief with inline citations** ...
{{else}}
Generate an intelligence briefing with the following sections: [full 7-section format]
{{/if}}

Lock file regenerated via ./gh-aw compile deep-report.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/graphql
- Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name k (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw er_test /sh ache/go/1.25.8/xTest User estl�� uypkflb2T --write ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet --ignore-path .prettierignore --log-level=erro-v ortcfg (http block)
https://api.github.com/orgs/test-owner/actions/secrets
- Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name on' --ignore-path ../../../.prettierignore --local bin/sh credential.usern/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
- Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 ache/go/1.25.8/x64/pkg/tool/linurepos/{owner}/{repo}/actions/runs/5/artifacts sv (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --get-regexp ^remote\..*\.gh-resolved$ er: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabl--show-toplevel npx prettier --wgh .cfg 64/pkg/tool/linu/repos/actions/github-script/git/ref/tags/v9 git rev-�� --git-dir 64/pkg/tool/linuremote.origin.url (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v5
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 /tmp/go-build3357502708/b151/vet.cfg 1/x64/bin/node (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git k/gh-aw/gh-aw/.ggit git /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel node /usr/bin/git /tmp/TestHashCongit git /usr/bin/git git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel infocmp /usr/bin/git f/tags/v6 ache/go/1.25.8/xrev-parse sv git rev-�� /ref/tags/v9 gh sv /repos/actions/ggit --jq /usr/bin/gh node (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260505-203229-14193/test-source-field-var-p rev-parse /usr/bin/infocmp (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --git-dir git /usr/bin/git */*.json' '!../.git --local x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git --local user.name x_amd64/compile git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv /tmp/go-build3357502708/b467/typ0 resolved$ /usr/bin/git -s -w -buildmode=exe git remo�� testing.testBinary=1 -extld=gcc (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v8
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv /repos/actions/setup-node/git/ref/tags/v4 --jq /usr/bin/git /ref/tags/v9 /tmp/go-build335rev-parse sv git rev-�� --show-toplevel git /usr/bin/git .github/workflowgit x_amd64/vet (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v9
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv on (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv /a.out (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v9.0.0
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv -c=4 -nolocalimports -importcfg /tmp/go-build3357502708/b394/importcfg -embedcfg /tmp/go-build3357502708/b394/embedcfg -pack tion�� on' --ignore-path ../../../.prettierignore --local x86_64/sh credential.usern/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv get --global n-dir/sh http.https://git/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel /opt/hostedtoolc--name-only /usr/bin/git 3229-14193/test-infocmp /tmp/go-build335-1 64/pkg/tool/linuxterm-color git rev-�� --show-toplevel 64/pkg/tool/linu--jq /usr/bin/git g_.a /tmp/go-build335rev-parse ache/go/1.25.8/x--show-toplevel git (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 /tmp/go-build3357502708/b453/_testmain.go sv rite '../../../*git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 /opt/hostedtoolcache/go/1.25.8/x--json sv 1926078928 /tmp/go-build335-1 e/git infocmp -1 xterm-color e/git /usr/bin/git OKEN }} tname) /opt/hostedtoolc--show-toplevel git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ithub-script/git/ref/tags/v9 git bject.type] | @tsv /tmp/TestGuardPogit rev-parse /usr/bin/gh git rev-�� /ref/tags/v9 gh sv /repos/actions/ginfocmp l 64/pkg/tool/linuxterm-color gh (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv go (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv ErrorFormatting4022799964/001 rev-parse /usr/bin/git l --local x_amd64/vet git rev-�� runs/20260505-203229-14193/test-1926078928 x_amd64/vet /usr/bin/git ithub/workflows user.name x_amd64/vet git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/vet r,url,status,conclusion,workflowDROP '**/*.ts' '**/*.git commit.gpgsign x_amd64/vet git conf�� --get l /usr/bin/git .js' --ignore-pagit (http block)
https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv kflow.test x_amd64/vet ortcfg.link */*.ts' '**/*.jsgit main nch,headSha,disp--show-toplevel uMRn_5pvPg4FQUB7He/88i7mEgrHQgazF3nzFAE/_AkwSfaH_x6mMwu8aC4W /tmp�� steps.test.outputs.result x_amd64/vet g_.a ify@v1.11.1/assegit ify@v1.11.1/asserev-parse x_amd64/compile e/git (http block)
https://api.github.com/repos/aws-actions/configure-aws-credentials/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git /tmp/TestGuardPogit rev-parse 7502708/b475/vet--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x--jq /usr/bin/gh /tmp/go-build335gh (http block)
- Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/node/24.14.1/x64/bin/node /usr/bin/git orce_all)' git repository(owne/repos/actions/github-script/git/ref/tags/v9 git rev-�� /ref/tags/v9 node sv /tmp/TestHashCongit x_amd64/vet /usr/bin/git gh (http block)
https://api.github.com/repos/azure/login/git/ref/tags/v2
- Triggering command: /usr/bin/gh gh api /repos/azure/login/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git /tmp/TestGuardPogit remote /usr/bin/git git rev-�� --show-toplevel git /usr/bin/infocmp mpleWorkflow3366infocmp x_amd64/vet /usr/bin/git infocmp (http block)
https://api.github.com/repos/docker/login-action/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/docker/login-action/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --show-toplevel node /usr/bin/git /tmp/TestHashCongit git /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git --get remote.origin.ur-1 om/upstream/repoxterm-color git (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --show-toplevel /opt/hostedtoolcremote /usr/bin/git 3229-14193/test-gh /tmp/go-build335api bject.type] | @t/repos/actions/github-script/git/ref/tags/v9 git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git '/tmp/TestParseinfocmp '/tmp/TestParse-1 clusion,workflowxterm-color git (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv (http block)
https://api.github.com/repos/github/gh-aw/actions/runs
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-28 (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-05 (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-02-04 **/*.json --ignore-path 64/pkg/tool/linuremote.origin.url eWit�� te 'scripts/**/*.js' --ignore-path .prettierignore --log-level=error .cfg 64/pkg/tool/linux_amd64/vet tierignore (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name on 64/pkg/tool/linux_amd64/vet /../../.prettier/usr/bin/git erignore (http block)
- Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name ../../../**/*.json x_amd64/compile --ignore-path ../../../.prettirun (http block)
- Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 on ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /../../.prettier/usr/bin/git erignore (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/1234567890
- Triggering command: /usr/bin/gh gh api repos/{owner}/{repo}/actions/runs/1234567890 --jq {databaseId: .id, number: .run_number, url: .html_url, status: .status, conclusion: .conclusion, workflowName: .name, workflowPath: .path, createdAt: .created_at, startedAt: .run_started_at, updatedAt: .updated_at, event: .event, headBranch: .head_branch, 15749813/001' 15749813/001' es/.bin/sh credential.usern/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/link (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name rkflow/js/**/*.json x_amd64/vet erignore ../../../.pretti-1 (http block)
- Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 on .cfg /../../.prettierinfocmp erignore th: .path, creatxterm-color ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name on ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /../../.prettier/opt/hostedtoolcache/node/24.14.1/x64/bin/node erignore (http block)
- Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet ignore-path ../.git b/gh-aw/actions/rev-parse (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /../../.prettiergit erignore (http block)
- Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg ger.test ignore-path ../.git (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name on x_amd64/compile /../../.prettiergit erignore (http block)
- Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet ignore-path ../.git (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg x_amd64/link /../../.prettiergit erignore (http block)
- Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg ache/go/1.25.8/x64/pkg/tool/linu-test.short=true ignore-path ../.git (http block)
https://api.github.com/repos/github/gh-aw/actions/workflows
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path on' --ignore-path ../../../.prettierignore --local ache/go/1.25.8/x64/bin/git credential.usern/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 ../../../.prettiremote (http block)
https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
- Triggering command: /tmp/go-build3357502708/b404/cli.test /tmp/go-build3357502708/b404/cli.test -test.testlogfile=/tmp/go-build3357502708/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel /opt/hostedtoolcache/go/1.25.8/x--jq (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv 3268187640/001 sRemoteWithRealGitmaster_branch4090847378/002/work .cfg **/*.json --ignore-path ../../../.prettixterm-color ache/go/1.25.8/x64/pkg/tool/linuconfig -c pkg/mod/github.com/aymanbagabas/remote.origin.url pkg/mod/github.com/aymanbagabas/go-udiff@v0.4.1/lcs/doc.go 64/pkg/tool/linux_amd64/vet w/js/**/*.json' infocmp (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv 1126187950/.github/workflows HEAD x_amd64/vet (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv 1126187950 HEAD x_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv */*.ts' '**/*.json' --ignore-path ../../../.pret.prettierignore HEAD x_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv */*.ts' '**/*.json' --ignore-path ../../../.pret.prettierignore main x_amd64/vet (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv */*.ts' '**/*.json' --ignore-path ../../../.pret.prettierignore HEAD x_amd64/vet (http block)
https://api.github.com/repos/google-github-actions/auth/git/ref/tags/v2
- Triggering command: /usr/bin/gh gh api /repos/google-github-actions/auth/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git /tmp/TestGuardPogit rev-parse /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel node /usr/bin/git /tmp/TestHashCongit x_amd64/vet /usr/bin/git git (http block)
https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
- Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv --write **/*.cjs ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet **/*.json --ignore-path ../../../.pretti--get-regexp ache/go/1.25.8/x^remote\..*\.gh-resolved$ -c 3229-14193/test-1304038138/.github/workflows (http block)
https://api.github.com/repos/nonexistent/repo/actions/runs/12345
- Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion ./../.prettieriggit gh/go-spew/spew (http block)
https://api.github.com/repos/owner/repo/actions/workflows
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x86_64/sh credential.usern/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo nfig/composer/ve-nilfunc credential.usern/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
- Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state .cfg **/*.ts b/gh-aw/cmd/gh-arev-parse --ignore-path ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
https://api.github.com/repos/test-owner/test-repo/actions/secrets
- Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name on' --ignore-pat-c=4 --local de_modules/.bin/-importcfg credential.helpe/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
https://api.github.com/repos/test/repo
- Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch t2148166484/.github/workflows (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/36fe2fc4-0a15-4c56-907c-1eecd2bb0b56 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

github-actions · 2026-05-05T21:18:44Z

Hey @copilot-swe-agent 👋 — great work implementing the output_format A/B experiment on deep-report.md! The frontmatter design (three variants, Mann-Whitney U analysis, guardrail threshold expressions) is well thought-out and the PR description is clear.

One thing that makes this hard to review:

Unfocused diff — the single meaningful change (deep-report.md) is buried inside 99 auto-regenerated .lock.yml updates for entirely unrelated workflows. This makes it very difficult for reviewers to spot-check what actually changed. Consider either (a) separating the lock file regeneration into a dedicated housekeeping PR, or (b) if lock regeneration is expected as part of this change, call that out explicitly in the PR body so reviewers know the lock files are intentional noise.

If you would like a hand cleaning this up, assign the following prompt to your coding agent:

Split PR #30447 in the github/gh-aw repository into two parts:
1. A focused PR containing only the deep-report.md change (the output_format A/B experiment).
2. A separate housekeeping PR containing only the 99 regenerated .lock.yml files.
If the lock files were regenerated as a direct side-effect of editing deep-report.md and are required to keep the repo consistent, add a section to the PR body of PR #30447 explaining this so reviewers can skip them during review.

Generated by Contribution Check · ● 758.5K · ◷

pelikhan · 2026-05-05T21:43:28Z

@copilot merge main and recompile

…periment-campaign # Conflicts: # .github/workflows/daily-cache-strategy-analyzer.lock.yml # .github/workflows/daily-fact.lock.yml # .github/workflows/daily-observability-report.lock.yml # .github/workflows/deep-report.lock.yml # .github/workflows/duplicate-code-detector.lock.yml # .github/workflows/grumpy-reviewer.lock.yml # .github/workflows/issue-arborist.lock.yml # .github/workflows/schema-feature-coverage.lock.yml # .github/workflows/smoke-call-workflow.lock.yml # .github/workflows/smoke-codex.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Adds an output_format experiment to the deep-report agentic workflow to A/B test alternative report structures aimed at reducing token cost while maintaining engagement, and refreshes multiple compiled workflow lockfiles to the latest pinned AWF container/image references.

Changes:

Added experiments.output_format rich-config frontmatter and variant-aware conditional report structure blocks in deep-report.md.
Updated numerous *.lock.yml compiled workflows to AWF v0.25.40 and pinned container digests / download lists.

Show a summary per file

File	Description
.github/workflows/deep-report.md	Adds the `output_format` experiment config and conditional prompt sections for variant-specific report structures.
.github/workflows/test-workflow.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/smoke-opencode.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/smoke-crush.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/smoke-ci.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/release.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/hippo-embed.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/firewall.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/example-permissions-warning.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/daily-malicious-code-scan.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/copilot-token-optimizer.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/codex-github-remote-mcp-test.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/code-simplifier.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/changeset.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/bot-detection.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/ai-moderator.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.
.github/workflows/ace-editor.lock.yml	Refreshes compiled lockfile to AWF v0.25.40 and pins updated container digests.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 52/215 changed files
Comments generated: 1

+experiments:
+  output_format:
+    variants: [full_briefing, executive_brief, annotated_brief]
+    description: "Tests whether report verbosity and structure affect token cost and discussion engagement"
+    hypothesis: "H0: no change in discussion engagement or token cost. H1: executive_brief reduces token usage by ≥20% without reducing engagement; annotated_brief improves actionability."
+    metric: token_count
+    secondary_metrics: [discussion_reactions, discussion_replies, output_char_length, run_duration_ms]
+    guardrail_metrics:
+      - name: empty_output_rate
+        threshold: "==0"
+      - name: issue_creation_success_rate
+        threshold: ">=0.8"
+    min_samples: 15
+    weight: [34, 33, 33]
+    start_date: "2026-05-06"
+    analysis_type: mann_whitney
+    tags: [output-format, token-cost, engagement, daily]


Copilot · 2026-05-05T21:50:52Z

@copilot merge main and recompile

Done in 115316f — merged origin/main and recompiled all workflows.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/graphql
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw Axa-ZHxP_hrM (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw owner=github -f git 1/x6�� add-source-path-451729645/.githu-s rev-parse e_modules/.bin/sh (http block)
https://api.github.com/orgs/test-owner/actions/secrets
- Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name h ../../../.prettierignore -f _modules/.bin/sh -f owner=github -f /usr/bin/gh api ./../pkg/workflo-errorsas -f me: String!) { -nilfunc -f owner=github -f node (http block)
https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
- Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv /repos/actions/setup-node/git/ref/tags/v6 --jq /usr/bin/git '/tmp/TestParseDgit '/tmp/TestParseDrev-parse /node_modules/.b--show-toplevel git rev-�� --show-toplevel node /usr/bin/git licyMinIntegritygh ../../../**/*.jsapi /usr/bin/infocmp/repos/actions/github-script/git/ref/tags/v9 git (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv /tmp/shared-actions-test2860552231 rev-parse (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v5
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv edOutput1000686315/001 on cfg.link nore erignore r: $owner, name:add 9k8-Gl3xy6m9Q9Uvmyorg -c t2599727759 git .yml get --local rostylelite-markuser.name node (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git --show-toplevel infocmp /usr/bin/gh git rev-�� --show-toplevel gh /usr/bin/git repos/{owner}/{rgit --jq /usr/bin/git git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git --show-toplevel node /usr/bin/git git rev-�� /ref/tags/v9 git sv --show-toplevel 1/x64/bin/node /usr/bin/git node (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --local --get on-checker.lock.yml (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --local --get mance.lock.yml (http block)
https://api.github.com/repos/actions/download-artifact/git/ref/tags/v8
- Triggering command: /usr/bin/gh gh api /repos/actions/download-artifact/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv on (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/download-artifact/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v8
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git ed } } git /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git /ref/tags/v9 test@example.comrev-parse sv git (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v9
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv pkg/actionpins/data/action_pins.json; \ cp .github/aw/actions-lock.json pkg/workflow/data/action_pins.json; \ echo "�� Action pins synced successfully"; \ else \ echo "�� (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv ." (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v9.0.0
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv js/**/*.json' --ignore-path ../../../.prettierignore (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/.github/workflows -f modules/@npmcli/run-script/lib/node-gyp-bin/node -f owner=github -f git -C json' --ignore-path ../../../.pr**/*.json show /usr/bin/infocmp (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9.0.0 --jq [.object.sha, .object.type] | @tsv rt (http block)
https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 1/x64/bin/node /usr/bin/gh == 'true' && 'fugh on rkflow/js/**/*.j/repos/actions/github-script/git/ref/tags/v9 gh api ref/tags/v1.2.3 repos/{owner}/{repo}/actions/runs/12345/artifacts sv .artifacts[].namgit bash ache/node/24.14.--show-toplevel git (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git json' --ignore-pgit show ules/.bin/node r conf�� ithub-script/git/ref/tags/v9 remote.origin.url bject.type] | @tsv th .prettierignogit rev-parse _modules/.bin/no--show-toplevel infocmp (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 1/x64/bin/node /usr/bin/git y on rkflow/js/**/*.j/repos/actions/github-script/git/ref/tags/v9 git rev-�� --show-toplevel sh /usr/bin/git 4950-12005/test-git bash /home/REDACTED/.ca--show-toplevel git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /tmp/TestCollectWorkflowFiles_WithImports1071384188/001 rev-parse /usr/bin/git -aw/git/ref/tagsgit infocmp bject.type] | @t--show-toplevel git rev-�� /ref/tags/v9 git sv /tmp/TestGuardPoinfocmp config /usr/bin/git infocmp (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --local --get ash (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen190516736/001' '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hyphen190516736/001' /node_modules/.bin/node tierignore (http block)
https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv /repos/actions/github-script/git/ref/tags/v9.0.0 --jq /usr/bin/gh 1 semgrep/semgrep:rev-parse ndor/bin/bash gh run list --json /usr/bin/git --workflow nonexistent-workrev-parse --limit git (http block)
https://api.github.com/repos/aws-actions/configure-aws-credentials/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel /usr/bin/git /usr/bin/git -aw/git/ref/tagsgit git (http block)
- Triggering command: /usr/bin/gh gh api /repos/aws-actions/configure-aws-credentials/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git /tmp/gh-aw-test-gh config /usr/bin/git git rev-�� /ref/tags/v9 git sv /tmp/TestGuardPogit remote e/git git (http block)
https://api.github.com/repos/azure/login/git/ref/tags/v2
- Triggering command: /usr/bin/gh gh api /repos/azure/login/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv --show-toplevel /usr/bin/git /usr/bin/git -v infocmp /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git /tmp/TestGuardPoinfocmp rev-parse /usr/bin/infocmpxterm-color git (http block)
https://api.github.com/repos/docker/login-action/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/docker/login-action/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --show-toplevel gh /usr/bin/git repos/{owner}/{rgit --jq /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git /tmp/TestGuardPoinfocmp remote /usr/bin/infocmpxterm-color git (http block)
https://api.github.com/repos/docker/metadata-action/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/docker/metadata-action/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv (http block)
https://api.github.com/repos/docker/setup-buildx-action/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/docker/setup-buildx-action/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ml (http block)
- Triggering command: /usr/bin/gh gh api /repos/docker/setup-buildx-action/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv (http block)
https://api.github.com/repos/github/gh-aw
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .default_branch /ref/tags/v9 --local k.yml user.name (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv om/github/gh-aw.git om/github/gh-aw.git /usr/bin/infocmp 4950-12005/test-git --write ache/node/24.14.--show-toplevel infocmp -1 xterm-color /opt/hostedtoolc--jq /usr/bin/git sistency_GoAndJainfocmp -importcfg /home/REDACTED/worxterm-color git (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv /ref/tags/v9 git sv tup-buildx-actiogit show odules/npm/node_--show-toplevel gh api /repos/actions/github-script/git/ref/tags/v9 --jq /usr/bin/docker on' --ignore-patgit (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv /tmp/gh-aw-test-runs/20260505-21.artifacts[].name remote /usr/bin/git k/gh-aw/gh-aw/.ggit (http block)
https://api.github.com/repos/github/gh-aw/actions/runs
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-28 ame erignore infocmp tion�� 729901570/.github/workflows git son ignore --global repository(owne--show-toplevel bash (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-05 ame erignore git tion�� ll-sweep (enforce_all)' rev-parse son ignore --global th: .path, creat/tmp/gh-aw-test-runs/20260505-214950-12005/test-1572006340 bash (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-02-04 ame erignore git tion�� te '../../../**/-f show 1/x64/bin/node ignore --global sv bash (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name (http block)
- Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmain_branch1308811248/001' sv s (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name bash ache/node/24.14.1/x64/bin/node --noprofile (http block)
- Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 **/*.cjs e/git **/*.json --ignore-path ../../../.pretti/repos/actions/github-script/git/ref/tags/v9 e/git /hom�� --write scripts/**/*.js /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/cgo .prettierignore --log-level=errorev-parse me: String!) { --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu--jq (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/1234567890
- Triggering command: /usr/bin/gh gh api repos/{owner}/{repo}/actions/runs/1234567890 --jq {databaseId: .id, number: .run_number, url: .html_url, status: .status, conclusion: .conclusion, workflowName: .name, workflowPath: .path, createdAt: .created_at, startedAt: .run_started_at, updatedAt: .updated_at, event: .event, headBranch: .head_branch, on' --ignore-path ../../../.prettierignore show modules/@npmcli/run-script/lib/node-gyp-bin/sh (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name bash sv l (http block)
- Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 n_pins.json; \ pkg/workflow/secure_markdown_rendering_test.go e/git k/gh-aw/gh-aw (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name bash che/go-build/d4/d41778fcc816302efc2f5156a970813dd497554df8010a573e374120b49e8e31-d ithub/workflows (http block)
- Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 git /opt/hostedtoolcache/uv/0.11.10/x86_64/node ithub/workflows set de node /opt�� runs/20260505-214950-12005/test-3137888135/.github/workflows --write (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name bash sv ithub/workflows (http block)
- Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 go ache/node/24.14.1/x64/bin/node s -s -w -X main.verev-parse /prettier ache/node/24.14.1/x64/bin/node 3647�� (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name bash sv ithub-script/gitgit (http block)
- Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 git /home/REDACTED/.local/bin/node k/gh-aw/gh-aw log t-go-checker.loc/repos/actions/github-script/git/ref/tags/v9 node /opt�� runs/20260505-214950-12005/test-3137888135/.github/workflows --write ache/node/24.14.1/x64/bin/node !../../../pkg/wogit --ignore-path ../../../.pretti--show-toplevel ache/node/24.14.1/x64/bin/node (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name bash /opt/hostedtoolcache/uv/0.11.10/x86_64/sh /ref/tags/v9 (http block)
- Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 git /home/REDACTED/go/bin/node =receive --merged=13ca40a-1 me: String!) { xterm-color (http block)
https://api.github.com/repos/github/gh-aw/actions/workflows
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path on' --ignore-path ../../../.prettierignore rev-parse me: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } l (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 ../../../.prettirev-parse (http block)
https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
- Triggering command: /tmp/go-build7360389/b404/cli.test /tmp/go-build7360389/b404/cli.test -test.testlogfile=/tmp/go-build7360389/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel git (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv iant-87955680/.github/workflows config tions/setup/js/node_modules/.bin/sh remote.origin.urgit (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv on' --ignore-path ../../../.pret.prettierignore config 1/x64/lib/node_modules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/node remote.origin.urgit (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv on' --ignore-pat--thin (http block)
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv on' --ignore-path ../../../.prettierignore --jq tions/node_modules/.bin/sh l (http block)
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv */*.ts' '**/*.json' --ignore-path ../../../.prettierignore -f ode_modules/.bin/sh -f owner=github -f git -C ../pkg/workflow/js/**/*.json' ---errorsas config kflows/static-analysis-report.lo-nilfunc remote.origin.urgit (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv Gitmain_branch1308811248/001' Gitmain_branch1308811248/001' node (http block)
https://api.github.com/repos/google-github-actions/auth/git/ref/tags/v2
- Triggering command: /usr/bin/gh gh api /repos/google-github-actions/auth/git/ref/tags/v2 --jq [.object.sha, .object.type] | @tsv graphql -f /usr/bin/git -f owner=github -f git rev-�� --show-toplevel git /usr/bin/git /ref/tags/v9 remote sv git (http block)
https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
- Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv matter-with-env-template-expressions.md rev-parse es/.bin/node l (http block)
https://api.github.com/repos/nonexistent/repo/actions/runs/12345
- Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion h ../../../.pretgit -f escape.lock.yml /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet -ato�� /ref/tags/v9 -buildtags sv l -ifaceassert -nilfunc infocmp (http block)
https://api.github.com/repos/owner/repo/actions/workflows
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo (http block)
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo /usr/bin/git l (http block)
- Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state /usr/bin/infocmp ithub/workflows (http block)
https://api.github.com/repos/test-owner/test-repo/actions/secrets
- Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name h ../../../.prettierignore rev-parse /usr/bin/git l (http block)
https://api.github.com/repos/test/repo
- Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch ithub-script/git/ref/tags/v9 --jq bject.type] | @tsv on' --ignore-patgit (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Initial plan

03ad514

Copilot AI assigned Copilot and pelikhan May 5, 2026

Copilot AI linked an issue May 5, 2026 that may be closed by this pull request

[ab-advisor] Experiment campaign for deep-report: A/B test output_format #30335

Closed

7 tasks

Copilot started work on behalf of pelikhan May 5, 2026 20:24 View session

Copilot AI and others added 2 commits May 5, 2026 20:30

initial plan for deep-report experiment campaign

b0a0092

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/36fe2fc4-0a15-4c56-907c-1eecd2bb0b56 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

add output_format experiment to deep-report workflow

13ca40a

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/36fe2fc4-0a15-4c56-907c-1eecd2bb0b56 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add A/B test for deep-report output format~~ [ab-advisor] Add output_format A/B experiment to deep-report workflow May 5, 2026

Copilot finished work on behalf of pelikhan May 5, 2026 20:38

Copilot AI requested a review from pelikhan May 5, 2026 20:38

github-actions Bot mentioned this pull request May 5, 2026

[Contribution Check Report] Contribution Check — 2026-05-05 #30250

Closed

pelikhan marked this pull request as ready for review May 5, 2026 21:43

Copilot AI review requested due to automatic review settings May 5, 2026 21:43

Copilot started work on behalf of pelikhan May 5, 2026 21:43 View session

Copilot started reviewing on behalf of pelikhan May 5, 2026 21:44 View session

Copilot AI reviewed May 5, 2026

View reviewed changes

Copilot finished work on behalf of pelikhan May 5, 2026 21:52

pelikhan merged commit 30f8866 into main May 5, 2026

pelikhan deleted the copilot/ab-advisor-experiment-campaign branch May 5, 2026 21:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ab-advisor] Add output_format A/B experiment to deep-report workflow#30447

[ab-advisor] Add output_format A/B experiment to deep-report workflow#30447
pelikhan merged 4 commits into
mainfrom
copilot/ab-advisor-experiment-campaign

Copilot AI commented May 5, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 5, 2026

Uh oh!

pelikhan commented May 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI commented May 5, 2026 •

edited

Loading

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Frontmatter changes

Prompt body changes

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

github-actions Bot commented May 5, 2026

Uh oh!

pelikhan commented May 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented May 5, 2026 •

edited

Loading

Copilot AI commented May 5, 2026 •

edited

Loading