Skip to content

fix(skill-optimizer): pre-flight stash, higher limits, targeted eval tasks#28292

Merged
pelikhan merged 3 commits intomainfrom
copilot/skill-optimizer-daily-improvements
Apr 24, 2026
Merged

fix(skill-optimizer): pre-flight stash, higher limits, targeted eval tasks#28292
pelikhan merged 3 commits intomainfrom
copilot/skill-optimizer-daily-improvements

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 24, 2026

Daily skill-optimizer runs were failing with target.repoPath: target repo has uncommitted changes (optimize.requireCleanGit is enabled) because init modifies workspace files before run checks for a clean tree. With only 10 tasks and 1 iteration, successful runs had almost no chance of reaching the 80% pass-rate target anyway.

Changes

Fix requireCleanGit failure (belt-and-suspenders)

  • Added git stash --include-untracked || true before the optimizer run and git stash pop || true (if: always()) after
  • Set "requireCleanGit": false in .skill-optimizer/skill-optimizer.json to handle cases where the stash approach isn't sufficient

Increase optimizer limits

  • maxTasks: 10 → 20
  • maxIterations: 1 → 3
  • Updated both skill-optimizer.json and the workflow's inline answers.json

Add three targeted evaluation tasks (.skill-optimizer/task-*.json)

  • task-ansi-escape-prevention.json — ANSI escape codes corrupting compiled YAML
  • task-sanitized-outputs-migration.jsonneeds.activation.outputs.*steps.sanitized.outputs.* deprecation
  • task-skill-optimizer-clean-git.json — configuring the optimizer for CI (the failure being fixed)

Each task includes input, ideal, and criteria fields for scoring.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw (http block)
    • Triggering command: /usr/bin/gh gh repo view --json owner,name --jq .owner.login + "/" + .name 64/pkg/tool/linux_amd64/vet GOINSECURE 051169/b011/asm./tmp/test-process-1891514284.js ache/go/1.25.8/xSecret: ${{ secrets.TOKEN }} 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh repo view owner/repo env ternal/tools/generate-action-metadata/main.go GO111MODULE x_amd64/link GOINSECURE l/ascii 051169/b092/syma--show-toplevel x_amd64/link -c /ref/tags/v9 -ZkR/Y5KUpR6ZrQZn8hJV-ZkR sv -n1 b/gh-aw/pkg/actiremote --end-of-options-v hz/8-8vmLiYCmHH9yLNKNaz/ITCHFh6Rtest-branch (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json gset/set.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git ithout_min-integgit .cfg 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linutest@example.com /usr/bin/git GgAk/DJ3JnFq1XLtnode GO111MODULE ache/go/1.25.8/xinstall git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/compile /usr/bin/git -json GO111MODULE x_amd64/vet git conf�� user.name Test User /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv se 2616426/b189/vetnonexistent-workflow-12345 64/pkg/tool/linu--limit -p internal/oserrordiff -lang=go1.25 64/pkg/tool/linu--name-only -o g_.a -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/link -p crypto/ecdh -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuconfig /usr/bin/git se 2616426/b078/vet\n .cfg git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linu.github/workflows/test.md /usr/bin/git ai-moderator.md 2616426/b256/vetrev-parse ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel git /usr/bin/git -m initial commit /usr/bin/git git rev-�� --show-toplevel git /usr/bin/git /tmp/TestGuardPogit rev-parse /usr/bin/git git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v6
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq [.object.sha, .object.type] | @tsv ons-test923628728 rev-parse /tmp/go-build2312616426/b444/stats.test l GO111MODULE x_amd64/compile /tmp/go-build2312616426/b444/stats.test -tes�� -test.paniconexit0 l /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-test.short=true -test.timeout=10git -test.run=^Test -test.short=true--show-toplevel /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git st-3513437459/.ggit 2616426/b210/vetcommit 64/pkg/tool/linu-m git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/compile /usr/bin/git g_.a /tmp/go-build231rev-parse /opt/hostedtoolc--show-toplevel git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json age/common.go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json .go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linuremote.origin.url /usr/bin/infocmp g_.a EocudEgAi 64/pkg/tool/linu--show-toplevel infocmp -1 xterm-color 64/pkg/tool/linu--auto /usr/bin/gh 3601268880/.githgit GO111MODULE 64/pkg/tool/linu--show-toplevel gh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ithub-script/git/ref/tags/v9 k/gh-aw/gh-aw/pkg/styles/theme.go bject.type] | @tsv -json 1.5.0/internal/jrev-parse x_amd64/compile node /tmp�� mpleWorkflow848727839/001 l ache/node/24.14.1/x64/bin/node -json GO111MODULE x_amd64/vet ache/node/24.14.1/x64/bin/node (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv -bool -buildtags /usr/bin/gh -errorsas -ifaceassert -nilfunc gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts /usr/bin/git .artifacts[].namgit GO111MODULE 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv ry=1 -extld=gcc 2616426/b452/_pkg_.a -json GO111MODULE x_amd64/compile git remo�� runs/20260424-145911-35913/test-1751488831 x_amd64/compile /usr/bin/git -json GO111MODULE x_amd64/compile git (http block)
  • https://api.github.com/repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv user.email ings.cjs odules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/git cjs st/suppress-warn-o 64/bin/node forks.js rev-�� HEAD st/suppress-warnmain _modules/.bin/gi-lang=go1.25 HEAD -aw/aw-test-owne--experimental-import-meta-resolve k/gh-aw/gh-aw/ac--require st/dist/workers//home/REDACTED/work/gh-aw/gh-aw/actions/setup/js/node_modules/vitest/suppress-warnings.cjs (http block)
    • Triggering command: /usr/bin/gh gh api /repos/astral-sh/setup-uv/git/ref/tags/eac588ad8def6316056a12d4907a9d4d84ff7a3b --jq [.object.sha, .object.type] | @tsv . tions/setup/js/node_modules/vite--stdout $name) { hasDiscussionsEnabled } } d git tions/node_modul-m git init�� -q (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv user.email test@example.com /usr/bin/git e=false vZBL1k16k 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel -MlvCPy/t6M8s7Cm2xpu7MOJIw7R /usr/bin/infocmp rtcfg GO111MODULE 64/pkg/tool/linu--show-toplevel infocmp (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv ErrorFormatting148346350/001 x_amd64/asm /opt/hostedtoolcache/node/24.14.1/x64/bin/node -json GO111MODULE x_amd64/compile node /tmp�� runs/20260424-145911-35913/test-1751488831 x_amd64/compile /usr/bin/git 01 GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/.github/workflows/agent-performance-analyzer.md config /usr/bin/git remote.origin.urgit GO111MODULE x_amd64/compile git -C /tmp/gh-aw-test-runs/20260424-145911-35913/test-133721770 config /usr/bin/git remote.origin.urgit GO111MODULE x_amd64/link git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD 051169/b007/syma--show-toplevel 64/pkg/tool/linux_amd64/vet env til.go til_test.go ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE b/gh-aw/pkg/giturev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 .cfg x_amd64/vet GOINSECURE boring GOMODCACHE x_amd64/vet env 051169/b162/_pkg_.a 3NxN/fOrMapTM_SttVIFB3NxN ache/go/1.25.8/x64/pkg/tool/linu-lang=go1.25 GOINSECURE th2/internal GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-test.v=true (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE 051169/b092/ GOMODCACHE 64/pkg/tool/linuTest User env 09/001/test-simple-frontmatter.md GO111MODULE .cfg GOINSECURE l 051169/b092/syma--git-dir ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 om/modelcontextprotocol/go-sdk@v1.5.0/internal/x-ifaceassert 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 051169/b233/_pkg_.a 3zY_/HcUWNrRjpCKdAR9m3zY_ .cfg GOINSECURE a95/uritemplate/config GOMODCACHE ache/go/1.25.8/xremote.origin.url (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE x_amd64/link GOINSECURE /execenv GOMODCACHE x_amd64/link env 051169/b224/_pkg_.a vNkW/MmwpPo_3e3tB-Au8vNkW .cfg GOINSECURE able GOMODCACHE dI/VlqCrFGhKv_vu-buildtags (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE l/buffer GOMODCACHE 64/pkg/tool/linux_amd64/vet ortc�� 09/001/test-simple-frontmatter.md stmain.go 64/pkg/tool/linux_amd64/vet GOINSECURE contextprotocol/rev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD 051169/b013/syma--show-toplevel 64/pkg/tool/linux_amd64/vet env 051169/b245/_pkg_.a 7Ps3/Xuna8G_bMUX3GMM57Ps3 64/pkg/tool/linux_amd64/compile GOINSECURE t/internal/strinconfig GOMODCACHE 64/pkg/tool/linutest@example.com (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140only GOMODCACHE 64/pkg/tool/linux_amd64/vet env 09/001/test-frontmatter-with-env-template-expressions.md wyMD/ZnqvKWWFy1YdeRMpwyMD ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE l/httpcommon GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 2235195712 fWCy/na03iXLzDBM34i--fWCy ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE /semver GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuremote.origin.url (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 051169/b021/atomrev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env 051169/b235/_pkg_.a DUdE/2oEXO76xEThYfB4YDUdE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE osh-tekuri/jsonsrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 2235195712 GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/aes/gcm GOMODCACHE 64/pkg/tool/linux_amd64/vet env 051169/b225/_pkg_.a ho52/RILG8Ja3npv64jHUho52 ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE ce GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg x_amd64/vet GOINSECURE GOMOD 051169/b029/syma--show-toplevel x_amd64/vet env 051169/b194/_pkg_.a Kv-X/SrddFjc3EqPBzwz7Kv-X ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-buildtags (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -c=4 -nolocalimports -importcfg /tmp/go-build2312616426/b423/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/gitutil/gitutil_test.go env -json o x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE x_amd64/vet env _.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 (http block)
  • https://api.github.com/repos/github/gh-aw/contents/.github/workflows/shared/reporting.md
    • Triggering command: /tmp/go-build2312616426/b404/cli.test /tmp/go-build2312616426/b404/cli.test -test.testlogfile=/tmp/go-build2312616426/b404/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/xrepos/{owner}/{repo}/actions/runs/2/artifacts /usr/bin/git agent-persona-exgit 2616426/b212/vetrev-parse ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git /tmp/go-build207ls -trimpath 1/x64/bin/node git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv 09/001/test-frontmatter-with-arr-errorsas 2616426/b006/vet.cfg ache/go/1.25.8/x64/pkg/tool/linu-nilfunc GOINSECURE 051169/b133/aritrev-parse ache/go/1.25.8/x--show-toplevel ache/go/1.25.8/x64/pkg/tool/linu-tests (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv /lib/wasm/wasm_e-p GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet 5087�� _.a @v1.19.2/ast/ast-nolocalimports x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet er_b�� _.a @v1.19.2/printer/color.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json @v1.1.3/cpu/cpuid/cpuid.go x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env _.a @v1.1.3/keyset/keyset.go x_amd64/vet GOINSECURE l/unsafebytes GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env phen164517034/001 phen164517034/002/work x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE 0Ji0fapdNGIB env 408550269/001 408550269/002/work x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv se 2616426/b013/vet.cfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linuconfig 0511�� /tmp/go-build207051169/b094/_pkgremote.origin.url k/gh-aw/gh-aw/pkg/logger/doc.go .cfg -p encoding -lang=go1.25 ache/go/1.25.8/x12345 (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE e/jsonschema-go/rev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE uVsL8NFDM1C9 env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --repo owner/repo --json name,path,state x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile -V=f�� 051169/b083/importcfg -Eee/499QsILxkBjFfa_H-Eee .cfg GOSUMDB GOWORK 64/bin/go ache/go/1.25.8/x64/pkg/tool/linuTest User (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
  • https://api.github.com/repos/test/repo
    • Triggering command: /usr/bin/gh gh api /repos/test/repo --jq .default_branch 051169/b194/_pkg_.a Kv-X/SrddFjc3EqPBzwz7Kv-X ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-buildtags ranc�� se 2616426/b062/vet.cfg .cfg -p internal/runtimerun -lang=go1.25 ache/go/1.25.8/x1 (http block)
  • invalid.example.invalid
    • Triggering command: /usr/lib/git-core/git-remote-https /usr/lib/git-core/git-remote-https origin https://invalid.example.invalid/nonexistent-repo.git e/git init�� ndor/bin/git git ode_modules/.bin/git =receive test@example.com--git-dir=/tmp/bare-incremental-vgB3Wu /git (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Fix daily skill optimizer run failure and implement improvements fix(skill-optimizer): pre-flight stash, higher limits, targeted eval tasks Apr 24, 2026
Copilot AI requested a review from pelikhan April 24, 2026 15:09
@pelikhan pelikhan marked this pull request as ready for review April 24, 2026 15:14
Copilot AI review requested due to automatic review settings April 24, 2026 15:14
@github-actions github-actions Bot mentioned this pull request Apr 24, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the daily skill-optimizer workflow and configuration to avoid clean-git failures in CI, increase optimization limits, and add targeted evaluation tasks for common workflow/optimizer issues.

Changes:

  • Adds pre/post git stash steps to the daily skill-optimizer workflow and increases task/iteration limits.
  • Updates .skill-optimizer/skill-optimizer.json (including requireCleanGit: false) and aligns workflow inline answers.json limits.
  • Introduces three new .skill-optimizer/task-*.json evaluation tasks with input, ideal, and criteria.
Show a summary per file
File Description
.skill-optimizer/skill-optimizer.json Increases optimizer limits and disables clean-git guard via requireCleanGit: false.
.skill-optimizer/task-ansi-escape-prevention.json Adds an eval task about avoiding/detecting ANSI escapes in workflow YAML.
.skill-optimizer/task-sanitized-outputs-migration.json Adds an eval task about migrating deprecated needs.activation.outputs.* usage.
.skill-optimizer/task-skill-optimizer-clean-git.json Adds an eval task about fixing CI failures due to clean-git requirements.
.github/workflows/daily-skill-optimizer.md Adds stash/restore steps and bumps inline optimizer limits.
.github/workflows/daily-skill-optimizer.lock.yml Regenerated compiled workflow to reflect the source workflow changes.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

.github/workflows/daily-skill-optimizer.md:125

  • git stash pop || true will attempt to pop even when no stash entry was created by the earlier step, and could also pop an unrelated stash if additional stashes are created later in the job. Consider capturing whether a stash was actually created (and/or the specific stash ref) and only popping in that case to make the cleanup deterministic.
      - name: Restore stashed changes
        if: always()
        shell: bash
        run: |
          git stash pop || true
  • Files reviewed: 6/6 changed files
  • Comments generated: 3

Comment on lines +45 to +48
- name: Stash any uncommitted changes
shell: bash
run: |
git stash --include-untracked || true
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The git stash --include-untracked step happens before the optimizer’s init, but the reported clean-tree failure occurs because init itself modifies the workspace before run executes. If the intent is to keep the tree clean for run (when requireCleanGit is enabled), this stash needs to happen after init and immediately before the run invocation (or init needs to be redirected to a temp worktree) so the changes introduced by init don’t trigger the clean-git guard.

This issue also appears on line 121 of the same file.

Copilot uses AI. Check for mistakes.
{
"id": "ansi-escape-prevention",
"input": "I'm editing a compiled workflow YAML file in .github/workflows/ and some terminal output I pasted seems to have broken the YAML parser. How do I avoid ANSI escape codes in workflow YAML files, and how do I check for them?",
"ideal": "ANSI escape codes (e.g. \\x1b[31m, \\x1b[0m) are terminal color codes that break YAML parsing. To avoid them: (1) never copy-paste from colored terminal output into YAML files; use --no-color flags or pipe through `| cat` to strip colors before saving; (2) run `make recompile` to regenerate clean .lock.yml files from .md sources—the workflow compiler automatically strips ANSI codes during compilation via stringutil.StripANSI(); (3) detect existing ANSI codes with: `find .github/workflows -name '*.yml' -o -name '*.yaml' | xargs grep -P '\\x1b\\[[0-9;]*[a-zA-Z]'`; (4) after fixing the source .md file, always run `make recompile` rather than hand-editing .lock.yml files.",
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Phrasing nit: piping through | cat doesn’t actually strip ANSI escape sequences; it just forces non-TTY output, which often disables auto-color for commands that honor it. To avoid misleading guidance in the task’s ideal answer, consider rewording this to “force non-TTY output to disable auto-color” (or suggest an explicit ANSI-stripping filter if you truly mean stripping).

Suggested change
"ideal": "ANSI escape codes (e.g. \\x1b[31m, \\x1b[0m) are terminal color codes that break YAML parsing. To avoid them: (1) never copy-paste from colored terminal output into YAML files; use --no-color flags or pipe through `| cat` to strip colors before saving; (2) run `make recompile` to regenerate clean .lock.yml files from .md sources—the workflow compiler automatically strips ANSI codes during compilation via stringutil.StripANSI(); (3) detect existing ANSI codes with: `find .github/workflows -name '*.yml' -o -name '*.yaml' | xargs grep -P '\\x1b\\[[0-9;]*[a-zA-Z]'`; (4) after fixing the source .md file, always run `make recompile` rather than hand-editing .lock.yml files.",
"ideal": "ANSI escape codes (e.g. \\x1b[31m, \\x1b[0m) are terminal color codes that break YAML parsing. To avoid them: (1) never copy-paste from colored terminal output into YAML files; use --no-color flags or pipe through `| cat` to force non-TTY output and often disable auto-color before saving; (2) run `make recompile` to regenerate clean .lock.yml files from .md sources—the workflow compiler automatically strips ANSI codes during compilation via stringutil.StripANSI(); (3) detect existing ANSI codes with: `find .github/workflows -name '*.yml' -o -name '*.yaml' | xargs grep -P '\\x1b\\[[0-9;]*[a-zA-Z]'`; (4) after fixing the source .md file, always run `make recompile` rather than hand-editing .lock.yml files.",

Copilot uses AI. Check for mistakes.
Comment on lines +4 to +9
"ideal": "There are two complementary fixes: (1) Add requireCleanGit set to false in the optimize section of .skill-optimizer/skill-optimizer.json — this disables the clean-git guard so the optimizer can run even when the workspace has uncommitted files (e.g. from a previous make recompile or init step that modifies config files). (2) In the workflow YAML, add a git stash --include-untracked step before invoking the optimizer and a corresponding git stash pop step (with if: always()) after the run to restore any stashed changes. Using requireCleanGit: false is the simplest fix for CI environments where the checkout is ephemeral; adding the stash step is a belt-and-suspenders approach that also works if the flag is not available in your version of the tool.",
"criteria": [
"Identifies setting requireCleanGit: false in .skill-optimizer/skill-optimizer.json as a fix",
"Mentions the optimize section of the config file",
"Suggests git stash --include-untracked before the optimizer run",
"Mentions git stash pop (with always() condition) to restore changes afterward",
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the scenario described (init modifies the workspace, then run fails due to a dirty tree), “stash before invoking the optimizer” won’t keep the tree clean for run because init will re-introduce changes after the stash. If you want the stash approach to address the clean-git guard, the guidance should indicate stashing after init and immediately before run (or running init in a separate worktree/temp clone), then restoring afterward.

Suggested change
"ideal": "There are two complementary fixes: (1) Add requireCleanGit set to false in the optimize section of .skill-optimizer/skill-optimizer.json — this disables the clean-git guard so the optimizer can run even when the workspace has uncommitted files (e.g. from a previous make recompile or init step that modifies config files). (2) In the workflow YAML, add a git stash --include-untracked step before invoking the optimizer and a corresponding git stash pop step (with if: always()) after the run to restore any stashed changes. Using requireCleanGit: false is the simplest fix for CI environments where the checkout is ephemeral; adding the stash step is a belt-and-suspenders approach that also works if the flag is not available in your version of the tool.",
"criteria": [
"Identifies setting requireCleanGit: false in .skill-optimizer/skill-optimizer.json as a fix",
"Mentions the optimize section of the config file",
"Suggests git stash --include-untracked before the optimizer run",
"Mentions git stash pop (with always() condition) to restore changes afterward",
"ideal": "There are two complementary fixes: (1) Add requireCleanGit set to false in the optimize section of .skill-optimizer/skill-optimizer.json — this disables the clean-git guard so the optimizer can run even when the workspace has uncommitted files (e.g. from a previous make recompile or an init step that modifies config files). (2) If you want a stash-based workaround in the workflow YAML, do not stash before init; instead, run init first, then add a git stash --include-untracked step immediately before the optimizer run step, and a corresponding git stash pop step (with if: always()) afterward to restore any stashed changes. An alternative is to run init in a separate worktree or temporary clone. Using requireCleanGit: false is the simplest fix for CI environments where the checkout is ephemeral; the post-init stash approach is a belt-and-suspenders option if the flag is not available in your version of the tool.",
"criteria": [
"Identifies setting requireCleanGit: false in .skill-optimizer/skill-optimizer.json as a fix",
"Mentions the optimize section of the config file",
"Suggests git stash --include-untracked after init and immediately before the optimizer run",
"Mentions git stash pop (with always() condition) afterward to restore changes",

Copilot uses AI. Check for mistakes.
@pelikhan pelikhan merged commit 3de7b98 into main Apr 24, 2026
40 checks passed
@pelikhan pelikhan deleted the copilot/skill-optimizer-daily-improvements branch April 24, 2026 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[skill-optimizer] Daily Skill Optimizer Improvements - 2026-04-23

3 participants