Skip to content

perf: stabilize BenchmarkValidation by removing permission-warning/logging overhead#27197

Merged
pelikhan merged 2 commits intomainfrom
copilot/performance-fix-validation-regression
Apr 19, 2026
Merged

perf: stabilize BenchmarkValidation by removing permission-warning/logging overhead#27197
pelikhan merged 2 commits intomainfrom
copilot/performance-fix-validation-regression

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 19, 2026

BenchmarkValidation was reporting a large slowdown, but part of the measured time came from avoidable warning/success-message output during each benchmark iteration rather than core validation work. This change makes the benchmark input and compiler settings align with the validation path being measured.

  • Benchmark fixture alignment

    • Added missing issues: read permission in the benchmark workflow frontmatter so tools.github default toolset validation does not emit a missing-permission warning on every run.
  • Benchmark output suppression

    • Enabled quiet mode for the benchmark compiler instance to prevent per-iteration success-message emission from skewing timing.
  • Resulting benchmark shape

    • The benchmark now measures validation/compiler execution more directly, with less contamination from console I/O.
permissions:
  contents: read
  issues: read
  pull-requests: read
...
compiler := NewCompiler(WithNoEmit(true))
compiler.SetStrictMode(true)
compiler.SetQuiet(true)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name 64/pkg/tool/linux_amd64/vet GOINSECURE 6866859/b087/sig/tmp/test-process-2956655343.js ache/go/1.25.8/xActor: ${{ github.actor }}, Repo: ${{ github.repository }} 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env 6866859/b178/_pkg_.a go ache/go/1.25.8/x64/pkg/tool/linu-nilfunc GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env 6866859/b213/_pkg_.a t2Bi/LbyKJAzlPTfrrG8ct2Bi ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet for-�� 759755736/001 k16k/fLMkIGmGY3YvZBL1k16k .cfg GOSUMDB GOWORK 64/bin/go ache/go/1.25.8/x^remote\..*\.gh-resolved$ (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json gset/set.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE 1/x64/lib/node_mGOMODCACHE GOINSECURE GOMOD GOMODCACHE go env .js' --ignore-paGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv list --json /usr/bin/infocmp --workflow nonexistent-workrev-parse --limit infocmp -1 xterm-color 64/pkg/tool/linux_amd64/vet /usr/bin/git b/workflows GO111MODULE 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv list --json /usr/bin/infocmp --workflow nonexistent-workrev-parse --limit infocmp -1 xterm-color node /usr/bin/git --check scripts/**/*.js ache/go/1.25.8/x--show-toplevel git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv mpleWorkflow2305275909/001 -trimpath om/myorg/repo.git -p main -lang=go1.25 /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/link -o /tmp/go-build2240133467/b467/scripts.test -importcfg /usr/bin/git -s -w -buildmode=exe git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyBlockedUsersCommaSeparatedCompiledOutput2213694355/001 config /usr/bin/git remote.origin.urgit GO111MODULE 64/bin/go /usr/bin/git conf�� --get-regexp ^remote\..*\.gh-resolved$ /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json GO111MODULE 64/bin/go /opt/hostedtoolcache/node/24.14.1/x64/bin/node (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv go t.go 1/x64/bin/node -p sync -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url t-ha�� vaScript4080644834/001/test-empty-frontmatter.md -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -p crypto/internal/rev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linutest@example.com (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/link /usr/bin/git util.test 0133467/b027/vetrev-parse ck git rev-�� --show-toplevel -4GmfpWzsnidAKUU7I/p67f43WHEJUTs4-oiIa1/9KqfPfOCremote /usr/bin/git se 0133467/b094/vetrev-parse 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linuTest User /usr/bin/git runs/20260419-14git -buildtags kflows/test-expi--show-toplevel git rev-�� --show-toplevel git /usr/bin/git . l /usr/bin/git git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git agent-performancgit 0133467/b230/vetrev-parse ache/go/1.25.8/xHEAD git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuTest User /usr/bin/git _messages.go _messages_test.grev-parse sv git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git -json GO111MODULE cfg git rev-�� --show-toplevel go /usr/bin/git ExpressionCompilgit GO111MODULE 1/x64/bin/node git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json arm.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json g/catmsg.go x_amd64/compile GOINSECURE GOMOD bytealg/compare_-bool x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/vet /usr/bin/git -json Kt0zQSK0W 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linuremote /usr/lib/git-core/git-remote-https ortcfg .cfg 64/pkg/tool/linu--show-toplevel /usr/lib/git-core/git-remote-https (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/vet /usr/bin/git -h gh-aw.wasm |git GO111MODULE 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linurev-parse /usr/bin/git 6866859/b055/_pkgit .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --git-dir go /usr/bin/git -json GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/gh or.md GO111MODULE 64/bin/go gh (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv NwDEsRBzvSaOC4LWAo9g/NwDEsRBzvSaOC4LWAo9g (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv xterm-color GOPROXY /usr/bin/git GOSUMDB GOWORK ules/.bin/node git -C /tmp/gh-aw-test-runs/20260419-144143-76073/test-759372991/.github/workflows config /usr/bin/git remote.origin.urgit GO111MODULE t git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv inputs.version x_amd64/vet /usr/bin/git g_.a Bzwz7Kv-X 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linu/tmp/file-tracker-test783508616/test2.lock.yml /usr/bin/git RUoqI5yik .cfg 64/pkg/tool/linu--show-toplevel /usr/bin/git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --git-dir go /usr/bin/git 01 GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/gh -json GO111MODULE 64/bin/go gh (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv git-upload-pack '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitbranch_with_hremote.origin.urgit l /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/link -json GO111MODULE x_amd64/compile /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linuconfig -o runs/20260419-143515-50298/test-1261434264 -importcfg /usr/bin/git s/test.md -w -buildmode=exe git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv git-receive-pack '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmain_branch1690254642/001' (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv run 0133467/b443/_testmain.go /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu--limit --detach -c=4 -nolocalimports /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linuremote.origin.url -o /tmp/go-build2240133467/b440/_pkg_.a -trimpath /usr/bin/git -p main -lang=go1.25 git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv /tmp/TestHashConsistency_GoAndJavaScript99964082--workflow GOPROXY /usr/bin/git GOSUMDB GOWORK tions/node_modul--show-toplevel git rev-�� --show-toplevel go /usr/bin/git */*.ts' '**/*.jsgit GO111MODULE 64/pkg/tool/linu--show-toplevel git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 34/001/test-inlined-imports-enabled-with-body-content.md gNV_/-ERQMY_tDmUJytyNgNV_ eutil.test GOINSECURE contextprotocol/-d GOMODCACHE eutil.test (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE ntio/asm/base64 GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6866859/b243/_pkg_.a GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/langurev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name --check 64/bin/go **/*.ts **/*.json --ignore-path git -c y_with_repos=public_1066619057/001 log 64/pkg/tool/linux_amd64/compile --format=%H:%ct x_amd64/compile 64/bin/go 64/pkg/tool/linuorigin (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6866859/b206/_pkg_.a hxms/bWOB0OjYPOs06SIChxms .cfg GOINSECURE hpke GOMODCACHE ache/go/1.25.8/xremote.origin.url (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go node /hom�� 2450993550/.github/workflows scripts/**/*.js ache/go/1.25.8/x64/bin/go -d --write 64/bin/go go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE fips140 GOMODCACHE 64/pkg/tool/linutest@example.com env 34/001/test-frontmatter-with-env-template-expressions.md .cfg 64/pkg/tool/linux_amd64/compile GOINSECURE g/x/crypto/interrev-parse GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go node /hom�� --check scripts/**/*.js ache/go/1.25.8/x64/bin/go -d --write 64/bin/go go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name om/modelcontextprotocol/go-sdk@vgithub.com/github/gh-aw/pkg/stats 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 34/001/test-inlined-imports-enab-s 3zY_/HcUWNrRjpCKdAR9m3zY_ x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE light 6866859/b012/sym--show-toplevel 64/pkg/tool/linux_amd64/vet ache�� thub/workflows 28ie/dWadUuI3oiBsYAOo28ie ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/language GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuTest User (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name --check 64/bin/go **/*.ts **/*.json --ignore-path git conf�� y_with_repos=public_1066619057/0remote.origin.url sh 64/pkg/tool/linux_amd64/asm "prettier" --wrigit x_amd64/compile 64/bin/go 64/pkg/tool/linuTest User (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 34/001/test-frontmatter-with-arrays.md GO111MODULE x_amd64/compile GOINSECURE l 6866859/b092/symuser.email x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 6866859/b092/ 6866859/b021/symabis 64/pkg/tool/linux_amd64/vet env 6866859/b257/_pkg_.a GO111MODULE x_amd64/compile GOINSECURE 6866859/b092/asmrev-parse ache/go/1.25.8/x--show-toplevel x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name --check 64/bin/go **/*.ts **/*.json --ignore-path git -c y_with_repos=public_1066619057/001 log 64/pkg/tool/linux_amd64/compile -n1 --format=format:config --end-of-optionsuser.email 64/pkg/tool/linutest@example.com (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 34/001/test-inlined-imports-enab-p aMu6/n6X7R7Av3bGkLZAPaMu6 x_amd64/compile GOINSECURE l/ascii GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE g/x/net/dns/dnsmrev-parse 6866859/b015/sym--show-toplevel 64/pkg/tool/linux_amd64/vet env thub/workflows jfLv/0caWgwAWMGdke8fejfLv ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE v3 GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go git -c 3/001/inlined-b.md log 64/pkg/tool/linux_amd64/link -n1 --format=format:rev-parse --end-of-options--show-toplevel 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name om/modelcontextprotocol/go-sdk@v-test.run=^Test 64/pkg/tool/linux_amd64/vet GOINSECURE l/buffer GOMODCACHE 64/pkg/tool/linux_amd64/vet env 34/001/test-inlined-imports-enabled-with-env-template-expressions-in-body.md pRaw/gwkwek_UF5vdtNyzpRaw util.test GOINSECURE contextprotocol/rev-parse GOMODCACHE util.test (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/bigmod 6866859/b015/sym--show-toplevel 64/pkg/tool/linux_amd64/vet env thub/workflows c9ZF/KtTFKQuDD_Pbt7zDc9ZF ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linutest@example.com (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name --check 64/bin/go **/*.ts **/*.json --ignore-path git for-�� --format=%(refname) n.go 64/pkg/tool/linux_amd64/link "prettier" --wrigit x_amd64/compile 64/bin/go 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -c=4 -nolocalimports -importcfg /tmp/go-build2240133467/b422/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil_test.go env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/compile env 6866859/b190/_pkg_.a ijPK/XGaGIDA5hxg1y3cTijPK 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github%2Fworkflows%2Faudit-workflows.md
    • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node --experimental-import-meta-resolve --require /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/suppress-warnings.cjs --conditions node --conditions development /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/dist/workers/forks.js -- t git show�� --verify --quiet k/gh-aw/gh-aw/actions/setup/js/ndevelopment --local --get de_modules/.bin/-e git (http block)
    • Triggering command: /opt/hostedtoolcache/node/24.14.1/x64/bin/node /opt/hostedtoolcache/node/24.14.1/x64/bin/node --experimental-import-meta-resolve --require /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/suppress-warnings.cjs --conditions node --conditions development /home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/dist/workers/forks.js -- e/git git show�� --verify --quiet ules/.bin/git 3 git t git (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build2240133467/b403/cli.test /tmp/go-build2240133467/b403/cli.test -test.testlogfile=/tmp/go-build2240133467/b403/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/asm env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /tmp/go-build482109704/b403/cli.test /tmp/go-build482109704/b403/cli.test -test.testlogfile=/tmp/go-build482109704/b403/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env '**/*.ts' '**/*.GOINSECURE GO111MODULE 64/bin/sh GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuconfig /usr/bin/git internal/abi pkg/mod/github.crev-parse .cfg git rev-�� --show-toplevel ache/go/1.25.8/xREDACTED /usr/bin/git Onlyrepos_only_wls 0133467/b259/vet-lh ache/go/1.25.8/x/tmp/gh-aw/aw-feature-branch.patch git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git -json GO111MODULE ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE ache/go/1.25.8/x/tmp/gh-aw/aw-feature-branch.patch git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv 6866859/b001/_pkg_.a GO111MODULE ache/go/1.25.8/x64/pkg/tool/linu-lang=go1.25 GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-dwarf=false (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE test GOINSECURE GOMOD GOMODCACHE test 8210�� -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a @v1.19.2/internal/format/format.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env lGitmain_branch1690254642/001' lGitmain_branch1690254642/001' 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD arith_wasm.s x_amd64/vet env g_.a GO111MODULE x_amd64/vet GOINSECURE id GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env g_.a @v1.19.2/context.go x_amd64/vet GOINSECURE nal/alias GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv nts.s emclr_wasm.s x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet 7035�� g_.a oding@v0.5.4/json/codec.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet om_b�� g_.a rt.go x_amd64/vet GOINSECURE obyte/asn1 GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env heck '**/*.cjs' '**/*.ts' '**/*.GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv 783104621 7gve/JS7DQw3o9RuNG8R67gve ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE Q1K54cwaT-D8R/TPrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv b/workflows GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env -json GO111MODULE 1/x64/bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE ntio/asm/ascii 6866859/b047/sym--show-toplevel 64/pkg/tool/linux_amd64/vet env 783104621 6866859/b047/importcfg x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion **/*.json --ignore-path ../../../.pretti--show-toplevel go env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json age/compact/comp-ifaceassert x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo -nolocalimports -importcfg /tmp/go-build2240133467/b419/importcfg -pack /tmp/go-build2240133467/b419/_testmain.go env -json age.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch thub/workflows jfLv/0caWgwAWMGdke8fejfLv ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE v3 GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch mpiledOutput386023722/001 -trimpath 64/pkg/tool/linux_amd64/link -p main -lang=go1.25 64/pkg/tool/linux_amd64/link env -json GO111MODULE 1/x64/bin/node GOINSECURE GOMOD GOMODCACHE go (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git git conf�� --local --get ode_modules/.bin/git cal/bin/git git /git git add . git tions/setup/node_modules/.bin/git -M main bin/git git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Fix performance regression in validation step perf: stabilize BenchmarkValidation by removing permission-warning/logging overhead Apr 19, 2026
Copilot AI requested a review from pelikhan April 19, 2026 14:50
@pelikhan pelikhan marked this pull request as ready for review April 19, 2026 14:54
Copilot AI review requested due to automatic review settings April 19, 2026 14:54
@pelikhan pelikhan merged commit 6e02b86 into main Apr 19, 2026
20 checks passed
@pelikhan pelikhan deleted the copilot/performance-fix-validation-regression branch April 19, 2026 14:54
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces benchmark noise/overhead so BenchmarkValidation measures compiler/validation work more directly (instead of console I/O from permission warnings and success messages).

Changes:

  • Add missing issues: read permission to benchmark workflow fixtures that use tools.github with the default toolset, preventing per-iteration missing-permission warnings.
  • Enable compiler quiet mode in BenchmarkValidation to suppress per-iteration success output.
Show a summary per file
File Description
pkg/workflow/compiler_performance_benchmark_test.go Adjusts benchmark fixtures and compiler settings to avoid permission-warning and success-message I/O during benchmark loops.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 1/1 changed files
  • Comments generated: 0

@github-actions github-actions bot mentioned this pull request Apr 19, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 70/100

⚠️ Acceptable — with context note

Metric Value
New/modified tests analyzed 2 (modified Benchmark* functions)
✅ Design tests (behavioral contracts) 2 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 0 — expected for benchmarks
Duplicate test clusters 0
Test inflation detected No (benchmark-only PR, no production changes)
🚨 Coding-guideline violations None

Test Classification Details

Test File Classification Issues Detected
BenchmarkCompileMemoryUsage (modified) pkg/workflow/compiler_performance_benchmark_test.go:168 ✅ Design Benchmark setup fix — adds missing issues: read permission to prevent spurious validation warnings that skew timing
BenchmarkValidation (modified) pkg/workflow/compiler_performance_benchmark_test.go:261 ✅ Design Adds issues: read + SetQuiet(true) to measure pure compilation time without I/O/logging overhead

Analysis Notes

i️ Benchmark-only PR: This PR modifies only Benchmark* functions, not Test* functions. The Test Quality Sentinel framework is primarily designed for Test* functions; the scoring is adapted accordingly.

Why the edge-case score is 0/30 (and that's fine here):
Benchmark functions (Benchmark*) are designed to measure performance of the happy path — adding error-case loops would pollute benchmark results and is explicitly out of scope for this PR. This is a framework-category mismatch, not a test-quality deficiency.

What the changes do:

  • BenchmarkCompileMemoryUsage: adds issues: read to the workflow YAML fixture so that a tools.github workflow has its minimal required permissions, preventing permission-warning log output that could add non-deterministic I/O overhead to timing measurements.
  • BenchmarkValidation: adds the same permission fix and calls compiler.SetQuiet(true) — the primary stabilization change, suppressing stderr/logging overhead to isolate pure validation CPU cost.

Both changes enforce the performance contract that benchmark results should measure compilation/validation work only, not incidental logging overhead.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 2 benchmarks — unit (//go:build !integration) ✅
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Verdict

Check passed. 0% of modified tests are implementation tests (threshold: 30%). No coding-guideline violations detected. Build tag (//go:build !integration) is present. No mock libraries used.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.


References: §24631880861

🧪 Test quality analysis by Test Quality Sentinel · ● 768.8K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 70/100. Test quality is acceptable — 0% of modified tests are implementation tests (threshold: 30%). This is a benchmark-only stabilization PR; the lower score (vs 80) reflects only the expected absence of error-case paths in benchmark functions, not a quality deficiency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[performance] Regression in Validation: 117.6% slower

3 participants