Fix thundering-herd rate-limit failures: spread FUZZY:DAILY pool from 3h to 18h window by Copilot · Pull Request #29369 · github/gh-aw

Copilot · 2026-04-30T18:48:12Z

On 2026-04-30, 10 workflow runs failed within 36 minutes because the FUZZY:DAILY scatter pool placed ~48% of all daily workflows in the 10:00–12:59 UTC window (GOOD tier, weight 2), saturating the GitHub App installation rate limit when 15–20 scheduled workflows fired concurrently.

Root cause

buildWeightedDailyPool() had three tiers:

BEST (02–05 UTC, weight 3): 72 slots
GOOD (10–12 UTC, weight 2): 300 slots ← 48% of the pool in 3 hours
OK (19–23 UTC, weight 1): 250 slots
Total: 622 slots, ~16% of workflows per hour in the 10–12 band

Change

Replace the three-tier design with two tiers, collapsing GOOD+OK into a single wide BROAD tier:

// Before
for h := 10; h <= 12; h++ {           // GOOD: 3 hours, weight 2
    for m := 5; m <= 54; m++ {
        pool = append(pool, timeSlot{h, m}, timeSlot{h, m})
    }
}
for h := 19; h <= 23; h++ { ... }     // OK: 5 hours, weight 1

// After
for h := 6; h <= 23; h++ {            // BROAD: 18 hours, weight 1
    for m := 5; m <= 54; m++ {
        pool = append(pool, timeSlot{h, m})
    }
}

New pool: 72 (BEST) + 900 (BROAD) = 972 slots. No single hour claims more than ~5% of the pool; the 6 workflows that previously clustered in 10:27–10:54 UTC now scatter across 06:37, 07:47, 08:23, 10:05, 15:12, 15:12.

Affected files

pkg/parser/schedule_fuzzy_scatter.go — new pool definition and updated doc comment with precise math
pkg/parser/schedule_parser_stability_test.go — updated expected cron values for FUZZY:DAILY and FUZZY:WEEKLY cross-platform consistency tests
pkg/parser/schedule_fuzzy_scatter_test.go — updated TestScatterScheduleUsesPreferredWindows from old BEST∪GOOD∪OK window check to [2, 23] (BEST∪BROAD)
90 workflow lock files recompiled with reshuffled schedules

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

https://api.github.com/graphql
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw fflib (http block)
- Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw ithub/workflows//tmp/fuzz-expr-1414147882.js ithub/workflows/github.prop.prop.prop.prop.prop.prop.prop.prop.prop.prop.prop.prop.prop.prop.proremote.origin.url p.prop.prop.pro ithu�� ned-imports-enabled-with-body-content.md ithub/workflows/changeset.lock.yml node (http block)
https://api.github.com/orgs/test-owner/actions/secrets
- Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name ithub/workflows (http block)
https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
- Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv ules/.bin/sh 64/pkg/tool/linutest@example.com /usr/bin/git 2170765740 -f /usr/bin/gh git rev-�� --show-toplevel /usr/bin/gh /usr/bin/git /ref/tags/v9 -f sv git (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v3
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv ts.result node /usr/bin/git ys.md ../../../**/*.jsrev-parse 1/x64/bin/npm git rev-�� --show-toplevel node (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v5
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv 0017-18070/test-.artifacts[].name config ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet remote.origin.urgh 0352666/b087/vetrepo e56243de2b6cca22view ache/go/1.25.8/xowner/repo -C /home/REDACTED/work/gh-aw/gh-aw/.github/workflows config 64/pkg/tool/linux_amd64/vet remote.origin.urgit 0352666/b115/vetrev-parse ules/.bin/node 64/pkg/tool/linux_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel ortcfg /usr/bin/git om/charmbraceletgit show ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git 0017-18070/test-git show 1/x64/bin/node git (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv tags/v4 git sv /tmp/TestGuardPogit config /opt/hostedtoolc--show-toplevel git rev-�� --show-toplevel node /usr/bin/git ub/gh-aw.git l ng.md git (http block)
https://api.github.com/repos/actions/checkout/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --get-regexp --local x_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv -unreachable=false son 1/x64/bin/node (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv 769023/b001/parser.test /tmp/go-build1640352666/b047/vet.cfg r: $owner, name: $name) { hasDiscussionsEnabled } } ignore (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v8
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel k/gh-aw/gh-aw/.github/workflows/bot-detection.lock.yml /usr/bin/git k/gh-aw/gh-aw/.ggit k/gh-aw/gh-aw/.gstatus clusion,workflowName,createdAt,s--show-toplevel git rev-�� --show-toplevel bash /usr/bin/git --noprofile /opt/hostedtoolcrev-parse /opt/hostedtoolc--show-toplevel git (http block)
https://api.github.com/repos/actions/github-script/git/ref/tags/v9
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -bool -buildtags ck.yml -errorsas -ifaceassert -nilfunc bash --no�� ithub/workflows -tests r: $owner, name: $name) { hasDiscussionsEnabled } } (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv d/gh-aw/main.go -buildtags ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -errorsas -ifaceassert -nilfunc ache/go/1.25.8/x64/pkg/tool/linuconfig -w -t security r: $owner, name: $name) { hasDiscussionsEnabled } } ./../.prettierig/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -d 168.63.129.16 ache/node/24.14.-trimpath (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv ithub/workflows -tests r: $owner, name: $name) { hasDiscussionsEnabled } } rkflow/js/**/*.jbash (http block)
https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git k/gh-aw/gh-aw config 64/pkg/tool/linu--show-toplevel git arch�� --remote=REDACTED v1.0.0 /usr/bin/git k/gh-aw/gh-aw config x_amd64/link git (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv -test.paniconexit0 s/1/artifacts /usr/bin/git -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel git init�� ed } } sh ache/node/24.14.1/x64/bin/node k/gh-aw/gh-aw k.go me: String!) { --show-toplevel ache/node/24.14.1/x64/bin/node (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --git-dir infocmp /usr/bin/gh LsRemoteWithRealgit LsRemoteWithRealrev-parse 64/pkg/tool/linu--show-toplevel gh run view 12345 /usr/bin/git nonexistent/repogit --json status,conclusio--show-toplevel git (http block)
https://api.github.com/repos/actions/setup-node/git/ref/tags/v6
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --get-regexp --local /sh (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv -unreachable=falremote.origin.url /tmp/go-build1640352666/b027/vet.cfg yml (http block)
- Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv --noprofile =my-default /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -unreachable=falgit /tmp/go-build164rev-parse $name) { has--show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linuremote.origin.url -o /tmp/go-build2587962411/b125/_pkg_.a -trimpath /usr/bin/infocmp -p github.com/githurev-parse -lang=go1.25 infocmp (http block)
https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
- Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/pkg/cli rev-parse /usr/bin/git l :latest /opt/hostedtoolc--show-toplevel git 7962�� mpleWorkflow3153833750/001 7962411/b462/_testmain.go /usr/bin/git k/gh-aw/gh-aw/.ggit --silent /bin/sh git (http block)
https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
- Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv js/**/*.json' --ignore-path ../../../.prettierignore .cfg 64/pkg/tool/linux_amd64/vet core.hooksPath (http block)
- Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv js/**/*.json' --ignore-path ../../../.prettierignore .cfg 64/pkg/tool/linux_amd64/vet copilot.original/opt/hostedtoolcache/node/24.14.1/x64/bin/node (http block)
https://api.github.com/repos/github/gh-aw
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw --jq .default_branch -unreachable=false rty 1/x64/bin/node (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git 24436360.go config 64/pkg/tool/linu--show-toplevel /usr/bin/git remo�� -v 64/pkg/tool/linuremote.upstream.url /usr/bin/git /home/REDACTED/worgit erena-mcp-serverrev-parse x_amd64/link git (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyTrustedUsersCompiledOutput2824090239/001 (http block)
https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv /tmp/TestGuardPolicyTrustedUsersCompiledOutput2824090239/001 rev-parse /usr/bin/git ithub/workflows (http block)
https://api.github.com/repos/github/gh-aw/actions/runs
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-04-23 --local 64/pkg/tool/linu--get x_amd64/compile api ithub/workflows -f /usr/bin/gh l owner=github -f /usr/bin/gh (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-03-31 --local ed } } /usr/bin/gh api ty-test.md -f age-analyzer.lock.yml -f owner=github ed } } /usr/bin/gh (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --limit 100 --created >=2026-01-30 --local 64/pkg/tool/linu../../../.prettierignore git -C ty-test.md est /usr/bin/git js/**/*.json' --node --local 64/pkg/tool/linu/home/REDACTED/work/gh-aw/gh-aw/.github/workflows/api-consumption-report.md git (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name 64/pkg/tool/linux_amd64/vet 64/pkg/tool/linux_amd64/compile rdian.md .cfg $name) { has--show-toplevel 64/pkg/tool/linux_amd64/compile api 300777268 -f /usr/bin/git -f owner=github -f git (http block)
- Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 64/pkg/tool/linux_amd64/vet 64/pkg/tool/linux_amd64/link json' --ignore-pgit .cfg DiscussionsEnabl--show-toplevel 64/pkg/tool/linux_amd64/link estl�� util.test show ortcfg.link res-fix-github-agit (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name config 64/pkg/tool/linux_amd64/compile l .cfg ed } } 64/pkg/tool/linu--json -C g_.a rev-parse er: String!, $na--created run /lcs ache/go/1.25.8/xgithub.token docker (http block)
- Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 64/pkg/tool/linux_amd64/vet 64/pkg/tool/linux_amd64/compile l .cfg 64/pkg/tool/linu--show-toplevel 64/pkg/tool/linux_amd64/compile api 2170765740 -f /usr/bin/git l olden -f git (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name go-udiff@v0.4.1/lcs/doc.go 64/pkg/tool/linux_amd64/compile --local HJ94ByTGM 64/pkg/tool/linuxterm-color 64/pkg/tool/linux_amd64/compile -C g_.a config /usr/bin/gh remote.origin.urgit tants $name) { hasDiscussionsEnabled } } /usr/bin/gh (http block)
- Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 rev-parse 64/pkg/tool/linux_amd64/vet plorer.md .cfg $name) { hasgithub.actor 64/pkg/tool/linux_amd64/vet -1 2170765740 64/pkg/tool/linurev-parse kflows/smoke-gemini.lock.yml th .prettierignogit k/gh-aw/gh-aw/pkconfig DiscussionsEnabluser.name git (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name -f 64/pkg/tool/linux_amd64/vet -f owner=github -f 64/pkg/tool/linux_amd64/vet -C 300777268/.github/workflows show /usr/bin/git th .prettierignogit k/gh-aw/gh-aw/pkrev-parse 86_64/bash git (http block)
- Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 config 64/pkg/tool/linux_amd64/compile l .cfg 64/pkg/tool/linuuser.name 64/pkg/tool/linuTest User -C rdian.md show /usr/bin/git l (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name ithub/workflows/instructions-janitor.lock.yml 64/pkg/tool/linux_amd64/link json' --ignore-pgit .cfg 64/pkg/tool/linu--git-dir 64/pkg/tool/linux_amd64/link -C 300777268/.github/workflows config ortcfg.link remote.origin.urgit k/gh-aw/gh-aw/pkrev-parse DiscussionsEnabl--show-toplevel UB4Z85HNraJd7B2VBq/aMwv_nfK75JUCd43tsyO/DAaZteIL-3lxG0HoYBiF (http block)
- Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 ithub/workflows/daily-fact.lock.yml /usr/bin/git json' --ignore-pgit .cfg 64/pkg/tool/linux_amd64/vet git -C k/gh-aw/gh-aw/.github/workflows config x_amd64/compile remote.origin.urgit (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name rev-parse 64/pkg/tool/linux_amd64/vet l .cfg eac68337d5455ec4user.name 64/pkg/tool/linuTest User -1 k/gh-aw/gh-aw/.github/workflows 64/pkg/tool/linushow x_amd64/vet l .cfg 64/pkg/tool/linu--get x_amd64/vet (http block)
- Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 -f 64/pkg/tool/linux_amd64/vet -f owner=github -f 64/pkg/tool/linux_amd64/vet -C 9/001/test-frontmatter-with-nested-objects.md config er: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnablxterm-color remote.origin.urgit (http block)
https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
- Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name config 64/pkg/tool/linux_amd64/compile remote.origin.urgit .cfg 64/pkg/tool/linuuser.email 64/pkg/tool/linutest@example.com -C 300777268/.github/workflows rev-parse x_amd64/compile th .prettierignogit k/gh-aw/gh-aw/pkconfig er.test x_amd64/compile (http block)
- Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 show 64/pkg/tool/linux_amd64/compile json' --ignore-pgit .cfg 64/pkg/tool/linuuser.email 64/pkg/tool/linutest@example.com -C 9/001/test-frontmatter-with-env--s rev-parse (http block)
https://api.github.com/repos/github/gh-aw/actions/workflows
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --noprofile (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 tigator.lock.ymlls-remote k.yml ck.yml --no�� checker.lock.yml ml bject.type] | @tsv ignore-path ../.git (http block)
- Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 owner=github -f 64/pkg/tool/linux_amd64/vet -1 k/gh-aw/gh-aw/.github/workflows 64/pkg/tool/linu-f /usr/bin/gh l main odules/npm/node_user.email /usr/bin/gh (http block)
https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
- Triggering command: /tmp/go-build2587962411/b404/cli.test /tmp/go-build2587962411/b404/cli.test -test.testlogfile=/tmp/go-build2587962411/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true -errorsas -ifaceassert -nilfunc bash k/gh�� --noprofile on anges-analyzer.lock.yml /../../.prettiergrep erignore 86_64/git sh (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv tags/v6 64/pkg/tool/linurev-parse sv 7962411/b390/_pkgit ache/go/1.25.8/xrev-parse 7962411/b390=> git rev-�� --show-toplevel git (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv 9/001/test-inlined-imports-enabl-p ithub/workflows/changeset.lock.ygithub.com/github/gh-aw/pkg/types_test .cfg ./../pkg/workflogit om/github/gh-aw n-dir/bash git -C /home/REDACTED/work/gh-aw/gh-aw/.g-c=4 rev-parse ache/go/1.25.8/x64/pkg/tool/linu-importcfg --noprofile (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv ithub/workflows (http block)
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv iant-2870051203 :latest /opt/hostedtoolcache/node/24.14.-f d -n 10 (http block)
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv ithub/workflows !../../../pkg/workflow/js/**/*.json /opt/hostedtoolcache/node/24.14.1/x64/bin/node ../../../.prettigit (http block)
https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
- Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/.g-c=4 (http block)
https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
- Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw show tartedAt,updatedAt,event,headBranch,headSha,displayTitle ./../pkg/workfloinfocmp = get && echo "p-1 ache/go/1.25.8/xxterm-color git ranc�� /home/REDACTED/work/gh-aw/gh-aw show ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet se 0352666/b079/vetrev-parse r: $owner, name:--show-toplevel ache/go/1.25.8/x64/pkg/tool/linuremote2 (http block)
https://api.github.com/repos/nonexistent/repo/actions/runs/12345
- Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion on' --ignore-patgit .cfg 64/pkg/tool/linu--show-toplevel eutil.test 5879�� k/gh-aw/gh-aw ghcr.io/github/serena-mcp-server:latest k ./../pkg/workflogit (http block)
https://api.github.com/repos/owner/repo/actions/workflows
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo sh ./../.prettierig/opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/vet ion_constants.go-atomic _test.go sh -c ithub/workflows ne_constants.go /opt/hostedtoolc-nilfunc ure_constants.gogh _constants.go test.go node (http block)
- Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo nfig/composer/vendor/bin/bash |human_friendly ion_constants.go-atomic _test.go bRjoFXLc7M9PD/wb-buildtags -c ry=1 ne_constants.go yml ure_constants.goinfocmp _constants.go test.go node (http block)
- Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state ortcfg.link th .prettierignogit k/gh-aw/gh-aw/pkconfig $name) { hasuser.email nt9zLENvjKcOLdPatest@example.com -C heckout/git/ref/tags/v6 o g_.a se 0352666/b033/vet-1 86_64/node git (http block)
https://api.github.com/repos/test-owner/test-repo/actions/secrets
- Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name ithub/workflows (http block)
https://api.github.com/repos/test/repo
- Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 0017-18070/test-source-field-variant-2870051203 ghcr.io/github/serena-mcp-server:latest k celain --ignore-git (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

The buildWeightedDailyPool function previously concentrated ~48% of all 'daily' workflows in a narrow 10:00–12:59 UTC window (GOOD tier, weight 2). This caused 10 workflow failures on 2026-04-30 when 15–20 workflows all triggered concurrently and exhausted the GitHub App installation rate limit. Change the pool from: BEST (02–05, weight 3) + GOOD (10–12, weight 2) + OK (19–23, weight 1) = 622 slots, ~48% in 10–12 UTC To: BEST (02–05, weight 3) + BROAD (06–23, weight 1) = 972 slots, ~5% per hour across 18 hours The per-hour density in the worst-case window drops from ~18 workflows/hour to ~1 workflow/hour for a typical fleet of 20 daily workflows. Update stability tests and preferred-window assertions to reflect the new distribution. Recompile all 205 workflow lock files with the new schedules. Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f010dbb7-0365-4194-9e2c-1d8a76e9d18d Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f010dbb7-0365-4194-9e2c-1d8a76e9d18d Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

github-actions · 2026-04-30T19:38:36Z

✅ smoke-ci: safeoutputs CLI comment + comment-memory run (25185462449)

Generated by Smoke CI for issue #29369 · ◷

github-actions · 2026-04-30T19:38:37Z

Comment Memory

CI lights the path
Green checks bloom at dawn
Quiet bots still sing

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Generated by Smoke CI for issue #29369 · ◷

Copilot

Pull request overview

Adjusts the FUZZY:DAILY scatter pool to reduce “thundering herd” scheduling in a narrow UTC window that was triggering GitHub App installation rate-limit failures.

Changes:

Replaces the previous BEST/GOOD/OK weighted daily pool with a BEST+BROAD design spanning 06:00–23:59 UTC.
Updates parser stability expectations and preferred-window tests to match the new pool.
Recompiles workflow lockfiles to apply reshuffled scattered cron schedules.

Show a summary per file

File	Description
pkg/parser/schedule_fuzzy_scatter.go	Redefines the weighted daily pool (BEST + BROAD) and updates explanatory comments/math.
pkg/parser/schedule_fuzzy_scatter_test.go	Updates preferred-window assertions/messages for full-day scatter patterns.
pkg/parser/schedule_parser_stability_test.go	Updates cross-platform expected cron outputs for FUZZY daily/weekly cases.
.github/workflows/workflow-skill-extractor.lock.yml	Updates scattered weekly cron time.
.github/workflows/workflow-normalizer.lock.yml	Updates scattered daily cron time.
.github/workflows/workflow-health-manager.lock.yml	Updates scattered daily cron time.
.github/workflows/weekly-safe-outputs-spec-review.lock.yml	Updates scattered weekly cron time.
.github/workflows/weekly-editors-health-check.lock.yml	Updates scattered weekly cron time.
.github/workflows/weekly-blog-post-writer.lock.yml	Updates scattered weekly cron time.
.github/workflows/update-astro.lock.yml	Updates scattered daily cron time.
.github/workflows/unbloat-docs.lock.yml	Updates scattered daily cron time.
.github/workflows/ubuntu-image-analyzer.lock.yml	Updates scattered weekly cron time.
.github/workflows/terminal-stylist.lock.yml	Updates scattered daily cron time.
.github/workflows/sub-issue-closer.lock.yml	Updates scattered daily cron time.
.github/workflows/step-name-alignment.lock.yml	Updates scattered daily cron time.
.github/workflows/static-analysis-report.lock.yml	Updates scattered daily cron time.
.github/workflows/spec-librarian.lock.yml	Updates scattered daily cron time.
.github/workflows/spec-extractor.lock.yml	Updates scattered daily cron time.
.github/workflows/spec-enforcer.lock.yml	Updates scattered daily cron time.
.github/workflows/smoke-ci.lock.yml	Updates scattered daily cron time.
.github/workflows/sergo.lock.yml	Updates scattered daily cron time.
.github/workflows/semantic-function-refactor.lock.yml	Updates scattered daily cron time.
.github/workflows/schema-consistency-checker.lock.yml	Updates scattered daily cron time.
.github/workflows/safe-output-health.lock.yml	Updates scattered daily cron time.
.github/workflows/refactoring-cadence.lock.yml	Updates scattered weekday cron time.
.github/workflows/prompt-clustering-analysis.lock.yml	Updates scattered daily cron time.
.github/workflows/metrics-collector.lock.yml	Updates scattered daily cron time.
.github/workflows/lockfile-stats.lock.yml	Updates scattered daily cron time.
.github/workflows/jsweep.lock.yml	Updates scattered daily cron time.
.github/workflows/issue-arborist.lock.yml	Updates scattered daily cron time (plus regenerated lock content).
.github/workflows/instructions-janitor.lock.yml	Updates scattered daily cron time.
.github/workflows/gpclean.lock.yml	Updates scattered daily cron time.
.github/workflows/go-logger.lock.yml	Updates scattered daily cron time.
.github/workflows/github-remote-mcp-auth-test.lock.yml	Updates scattered daily cron time.
.github/workflows/firewall-escape.lock.yml	Updates scattered daily cron time.
.github/workflows/duplicate-code-detector.lock.yml	Updates scattered daily cron time (plus regenerated lock content).
.github/workflows/draft-pr-cleanup.lock.yml	Updates scattered daily cron time.
.github/workflows/docs-noob-tester.lock.yml	Updates scattered daily cron time.
.github/workflows/developer-docs-consolidator.lock.yml	Updates scattered daily cron time.
.github/workflows/dependabot-burner.lock.yml	Updates scattered weekly cron time.
.github/workflows/delight.lock.yml	Updates scattered daily cron time.
.github/workflows/dead-code-remover.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-workflow-updater.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-token-consumption-report.lock.yml	Updates scattered weekday cron time.
.github/workflows/daily-testify-uber-super-expert.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-team-evolution-insights.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-syntax-error-quality.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-skill-optimizer.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-sentrux-report.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-semgrep-scan.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-security-red-team.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-security-observability.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-secrets-analysis.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-safe-outputs-conformance.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-safe-output-optimizer.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-safe-output-integrator.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-rendering-scripts-verifier.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-regulatory.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-performance-summary.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-otel-instrumentation-advisor.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-observability-report.lock.yml	Updates scattered daily cron time (plus regenerated lock content).
.github/workflows/daily-multi-device-docs-tester.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-malicious-code-scan.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-issues-report.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-function-namer.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-firewall-report.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-doc-healer.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-compiler-quality.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-community-attribution.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-code-metrics.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-cli-tools-tester.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-cli-performance.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-caveman-optimizer.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-cache-strategy-analyzer.lock.yml	Updates scattered daily cron time (plus regenerated lock content).
.github/workflows/daily-aw-cross-repo-compile-check.lock.yml	Updates scattered weekday cron time.
.github/workflows/daily-astrostylelite-markdown-spellcheck.lock.yml	Updates scattered daily cron time.
.github/workflows/daily-assign-issue-to-user.lock.yml	Updates scattered daily cron time.
.github/workflows/copilot-session-insights.lock.yml	Updates scattered daily cron time.
.github/workflows/copilot-pr-prompt-analysis.lock.yml	Updates scattered daily cron time.
.github/workflows/copilot-opt.lock.yml	Updates scattered weekly cron time.
.github/workflows/copilot-cli-deep-research.lock.yml	Updates scattered daily cron time.
.github/workflows/copilot-agent-analysis.lock.yml	Updates scattered daily cron time.
.github/workflows/constraint-solving-potd.lock.yml	Updates scattered daily cron time.
.github/workflows/code-simplifier.lock.yml	Updates scattered daily cron time.
.github/workflows/cli-version-checker.lock.yml	Updates scattered daily cron time.
.github/workflows/claude-code-user-docs-review.lock.yml	Updates scattered daily cron time.
.github/workflows/audit-workflows.lock.yml	Updates scattered daily cron time.
.github/workflows/api-consumption-report.lock.yml	Updates scattered daily cron time.
.github/workflows/agent-persona-explorer.lock.yml	Updates scattered daily cron time.
.github/workflows/agent-performance-analyzer.lock.yml	Updates scattered daily cron time.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 90/90 changed files
Comments generated: 1

Copilot · 2026-04-30T19:41:33Z

 // buildWeightedDailyPool constructs the weighted pool of (hour, minute) time slots
 // used for full-day scatter patterns. The pool reflects the following distribution:
 //
 //   - BEST  (weight 3): 02:00–05:59 UTC at odd minutes (07,13,23,37,43,53)
-//   - GOOD  (weight 2): 10:00–12:59 UTC (gap between EU/US peaks), minutes [5,54]
-//   - OK    (weight 1): 19:00–23:59 UTC (evening hours), minutes [5,54]
+//   - BROAD (weight 1): 06:00–23:59 UTC, minutes [5,54]
 //
-// Using weights means a randomly selected slot is 3× more likely to land in the
-// BEST window than the OK window.
+// Pool size: 4×6×3 (BEST) + 18×50×1 (BROAD) = 72 + 900 = 972 slots.
+// BEST represents 72/972 ≈ 7% and BROAD represents 900/972 ≈ 93% of slots.
+// Within BROAD, each hour claims 50/972 ≈ 5% of the pool.
+//
+// The BROAD tier spans the full daytime and evening window to prevent thundering-herd
+// API rate-limit bursts. The former design used a GOOD tier (10:00–12:59 UTC, weight 2)
+// that concentrated ~300/622 ≈ 48% of pool slots in a 3-hour window—equivalent to
+// ~16% of workflows per hour in that band. With BROAD, no single hour claims more
+// than ~5% of workflows, so 20 concurrent daily workflows spread across roughly


The repository’s fuzzy schedule specification doc is now out of sync with this new BROAD-tier pool. docs/src/content/docs/reference/fuzzy-schedule-specification.md still describes BEST/GOOD/OK with a 622-slot pool and related test bullets; please update it to the new BEST+BROAD tiers and 972-slot pool (including the worked example and any references to the old windows).

github-actions · 2026-04-30T19:41:51Z

🧪 Test Quality Sentinel Report

Test Quality Score: 95/100

✅ Excellent test quality

Metric	Value
New/modified tests analyzed	2
✅ Design tests (behavioral contracts)	2 (100%)
⚠️ Implementation tests (low value)	0 (0%)
Tests with error/edge cases	2 (100%)
Duplicate test clusters	0
Test inflation detected	No
🚨 Coding-guideline violations	None

Test Classification Details

Test	File	Classification	Issues Detected
`TestScatterScheduleUsesPreferredWindows`	`pkg/parser/schedule_fuzzy_scatter_test.go:815`	✅ Design	None — behavioral contract well-enforced
`TestScatterScheduleCrossPlatformConsistency`	`pkg/parser/schedule_parser_stability_test.go:49`	✅ Design	None — pins deterministic outputs for known inputs

Analysis Overview

Both changed test files contain only modifications to existing tests — no new test functions were added. The changes are exactly what this PR requires:

TestScatterScheduleUsesPreferredWindows — updated the isInPreferredWindow predicate from (hour >= 2 && hour <= 5) || (hour >= 10 && hour <= 12) || (hour >= 19 && hour <= 23) to hour >= 2 && hour <= 23 to reflect the new BROAD (06–23 UTC) window replacing the old GOOD+OK split. This is a behavioral contract test: it asserts that no schedule ever lands in the 00:01 UTC band, which is the core correctness guarantee of the thundering-herd fix. Runs 10 diverse workflow identifiers.
TestScatterScheduleCrossPlatformConsistency — updated expected cron values (e.g., "38 6 * * *" for workflow-a.md) to pin the new deterministic hash outputs under the enlarged pool. This is a cross-platform stability test: it catches any hash-function drift across OS/arch and ensures the scatter is reproducible. Table-driven with 8 scenarios covering daily, hourly, weekly, DAILY_AROUND, and WEEKLY_AROUND variants.

Score Breakdown

Component	Score	Notes
Behavioral Coverage (40 pts)	40	Both tests verify observable scheduler output
Error/Edge Case Coverage (30 pts)	25	Stability test lacks explicit error paths, but scatter test covers error cases in unchanged rows
Low Duplication (20 pts)	20	No duplicate clusters
Proportional Growth (10 pts)	10	Test additions (8 lines) vs production additions (20 lines) → ratio 0.4 (well under 2:1)

-5 pts: TestScatterScheduleCrossPlatformConsistency uses t.Errorf / t.Fatalf without descriptive assertion messages in some calls (bare format strings like ScatterSchedule(%q, %q) = %q, want %q are fine as format args but the sub-test name alone sometimes lacks context). Minor deduction.

Language Support

Tests analyzed:

🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Verdict

✅ Check passed. 0% of new/modified tests are implementation tests (threshold: 30%). Both modified tests enforce genuine behavioral contracts: the time-window invariant and deterministic cross-platform output of the FUZZY:DAILY scatter pool.

📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

Assert on observable outputs, return values, or state changes
Cover error paths and boundary conditions
Would catch a behavioral regression if deleted
Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

Assert on internal function calls (mocking internals)
Only test the happy path with typical inputs
Break during legitimate refactoring even when behavior is correct
Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

References: §25185463701

🧪 Test quality analysis by Test Quality Sentinel · ● 715.7K · ◷

github-actions

✅ Test Quality Sentinel: 95/100. Test quality is excellent — 0% of modified tests are implementation tests (threshold: 30%). Both updated tests enforce genuine behavioral contracts: the BROAD time-window invariant and deterministic cross-platform output of the FUZZY:DAILY scatter pool.

Initial plan

1c7349e

Copilot AI assigned Copilot and gh-aw-bot Apr 30, 2026

Copilot started work on behalf of gh-aw-bot April 30, 2026 18:48 View session

Copilot AI linked an issue Apr 30, 2026 that may be closed by this pull request

[aw-failures] Fix: GitHub App installation rate limit burst — 10 workflows failed in 36-minute window (2026-04-30 11:05–11:41 UTC) #29318

Closed

Copilot AI and others added 2 commits April 30, 2026 19:03

Clarify pool distribution comments per code review feedback

d9ff792

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/f010dbb7-0365-4194-9e2c-1d8a76e9d18d Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix GitHub App installation rate limit burst issue~~ Fix thundering-herd rate-limit failures: spread FUZZY:DAILY pool from 3h to 18h window Apr 30, 2026

Copilot finished work on behalf of gh-aw-bot April 30, 2026 19:09

Copilot AI requested a review from gh-aw-bot April 30, 2026 19:09

Merge branch 'main' into copilot/aw-failures-fix-github-app-rate-limit

493d44b

pelikhan marked this pull request as ready for review April 30, 2026 19:36

Copilot AI review requested due to automatic review settings April 30, 2026 19:36

pelikhan merged commit 93a4eec into main Apr 30, 2026
42 of 52 checks passed

pelikhan deleted the copilot/aw-failures-fix-github-app-rate-limit branch April 30, 2026 19:37

Copilot started reviewing on behalf of pelikhan April 30, 2026 19:37 View session

Copilot AI reviewed Apr 30, 2026

View reviewed changes

github-actions Bot mentioned this pull request Apr 30, 2026

[aw] No-Op Runs #29134

Open

github-actions Bot approved these changes Apr 30, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix thundering-herd rate-limit failures: spread FUZZY:DAILY pool from 3h to 18h window#29369

Fix thundering-herd rate-limit failures: spread FUZZY:DAILY pool from 3h to 18h window#29369
pelikhan merged 4 commits intomainfrom
copilot/aw-failures-fix-github-app-rate-limit

Copilot AI commented Apr 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

github-actions Bot commented Apr 30, 2026

Uh oh!

github-actions Bot commented Apr 30, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 30, 2026

Uh oh!

github-actions Bot commented Apr 30, 2026

Uh oh!

github-actions Bot left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Copilot AI commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Root cause

Change

Affected files

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

Uh oh!

github-actions Bot commented Apr 30, 2026

Uh oh!

github-actions Bot commented Apr 30, 2026

Comment Memory

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

Copilot AI Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Apr 30, 2026

🧪 Test Quality Sentinel Report

Test Quality Score: 95/100

Test Classification Details

Analysis Overview

Score Breakdown

Language Support

Verdict

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Apr 30, 2026 •

edited

Loading