Skip to content

Fix BYOK Copilot model fallback to avoid empty COPILOT_MODEL in compiled workflows#26566

Merged
lpcox merged 5 commits intomainfrom
copilot/fix-empty-copilot-model-fallback
Apr 16, 2026
Merged

Fix BYOK Copilot model fallback to avoid empty COPILOT_MODEL in compiled workflows#26566
lpcox merged 5 commits intomainfrom
copilot/fix-empty-copilot-model-fallback

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 16, 2026

byok-copilot workflows were compiling with COPILOT_MODEL fallback set to '', which breaks Copilot CLI BYOK mode because BYOK providers require an explicit model. This change ensures BYOK compilations always emit a non-empty fallback while preserving existing behavior for non-BYOK workflows.

  • BYOK model fallback behavior

    • Updated Copilot execution env generation to use a non-empty fallback when:
      • engine is Copilot
      • model is not explicitly configured
      • features.byok-copilot is enabled
    • Non-BYOK behavior remains unchanged (|| ''), so native defaults continue to apply there.
  • Constants

    • Added CopilotBYOKDefaultModel in engine constants:
      • claude-sonnet-4.6
  • Coverage updates

    • Extended TestCopilotFallbackModelMapsToNativeEnvVar to assert BYOK-specific fallback rendering for agent runs.
    • Existing detection/non-BYOK expectations remain intact.

Example of generated env line in BYOK mode:

COPILOT_MODEL: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || 'claude-sonnet-4.6' }}

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw 925117/b249/emberev-parse ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git 0005-32299/test-git -trimpath res.lock.yml git (http block)
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE .cfg git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE 5264560/b070/gh---show-toplevel git (http block)
  • https://api.github.com/orgs/test-owner/actions/secrets
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD ic/asm.s go (http block)
    • Triggering command: /usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name 2961917/b388/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env Jp80/wsXtYQS6C8KGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 2961917/b388/imp-trimpath (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linu-tests /opt/hostedtoolcache/node/24.14.1/x64/bin/node YnTZR3KAf .Core.RunCommandrev-parse 64/pkg/tool/linu--show-toplevel /opt/hostedtoolcache/node/24.14.1/x64/bin/node /tmp�� Safe: ${{ github.actor }}, Unsafe: ${{ secrets.TOKEN }} 64/pkg/tool/linu--auto /usr/bin/git 925117/b069/_pkggit .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq [.object.sha, .object.type] | @tsv --show-toplevel x_amd64/asm /usr/bin/gh -json GO111MODULE 64/bin/go gh run list --json /usr/bin/git --workflow nonexistent-workrev-parse --limit git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv G_es/lMB5IYBY6CLlpDj_G_es rev-parse ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet 1182294371/001' 1182294371/001' x_amd64/compile ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq [.object.sha, .object.type] | @tsv sistency_InlinedImports2684826178/001/noflag-a.md config /usr/bin/git remote.origin.urgit GO111MODULE 64/bin/go git -C /tmp/gh-aw-test-runs/20260416-051424-55290/test-2669543082 remote ache/node/24.14.1/x64/bin/node 01 GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv util.test 5114303/b099/vet.cfg ortcfg.link -I /tmp/go-build268run -I fEmuZxTCue0RDK6B12346 (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 925117/b218/_pkggit Ldjv/q8rDzC5dO2Krev-parse 64/pkg/tool/linu--show-toplevel git rev-�� it/ref/tags/v4 64/pkg/tool/linux_amd64/link sv util.test TwGo/SQWnFYXBmVnrev-parse ortcfg.link git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq [.object.sha, .object.type] | @tsv --show-toplevel /opt/hostedtoolcache/go/1.25.8/x-importcfg /usr/bin/git ring1017931150/0git /tmp/go-build235rev-parse Name,createdAt,s--show-toplevel git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.8/x-extld=gcc /usr/bin/git -bool -buildtags 1/x64/bin/node git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel ache/go/1.25.8/x64/pkg/tool/linu-trimpath /usr/bin/git se 5114303/b045/vetcommit .cfg git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linuorigin /usr/bin/git rdian.md 5114303/b221/vetrev-parse ache/go/1.25.8/x--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/link /usr/bin/git 62/001/test-frongit GO111MODULE ortcfg.link git rev-�� --show-toplevel 3ywv4CpMHBh6K-yK1 /usr/bin/git -json GO111MODULE g_.a git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json .go x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json o x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env go GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv = get && echo "******"; }; f get = get && echo "******"; }; f get /usr/bin/git rity882692381/00git GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git -json .cfg x_amd64/vet git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv = get && echo "******"; }; f get = get && echo "******"; }; f get /usr/bin/git -json GO111MODULE nch,headSha,disp--show-toplevel git rev-�� --show-toplevel go /usr/bin/git ub/workflows GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv image:v1.0.0 remote.myorg.url /usr/bin/git _.a GO111MODULE nch,headSha,disp--show-toplevel git conf�� user.email test@example.com /usr/bin/git -json .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv om/owner/repo.git om/owner/repo.git /usr/bin/git ub/workflows GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv t0 -importcfg 1/x64/bin/node m0s -w (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq [.object.sha, .object.type] | @tsv log.showsignature=false log /usr/bin/git -n1 --format=format:rev-parse --end-of-options--show-toplevel git clon�� /tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmaster_branch2776450696/001 /tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmaster_branch2776450696/002/work /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.1.2
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv GOMODCACHE x_amd64/vet /usr/bin/git repo4197299145/0git GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git DseGpepMC .cfg 64/pkg/tool/linu--show-toplevel git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.1.2 --jq [.object.sha, .object.type] | @tsv add upstream /usr/bin/git ub/workflows GO111MODULE 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE x_amd64/vet git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv sistency_GoAndJavaScript2686379451/001/test-complex-frontmatter-with-tools.md -tests (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv --end-of-options blob /usr/bin/git GOSUMDB GOWORK 64/bin/go git init�� :latest --initial-branch=develop 5264560/b427/vet.cfg -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv sistency_GoAndJavaScript2686379451/001/test-simple-frontmatter.md -trimpath ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile -p github.com/githurev-parse -lang=go1.25 ache/go/1.25.8/x64/pkg/tool/linux_amd64/compile work�� 5114303/b446/_pkg_.a --json 5114303/b446=> -c=4 -nolocalimports -importcfg git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv --format=%(refname) --merged=4544c2cbc62ff88042217578120dd9d14c65a837 /bin/sh GOSUMDB GOWORK 64/bin/go /bin/sh -c git-upload-pack '/tmp/TestParseDefaultBranchFromLsRemoteWithReal--detach git-upload-pack '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmaster_branch2776450696/001' /usr/bin/git sions.md GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE 925117/b015/chacrev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env 4002497977 DUdE/2oEXO76xEThYfB4YDUdE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/formarev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE /go-yaml/internarev-parse 925117/b047/syma--show-toplevel 64/pkg/tool/linux_amd64/vet env m_exec.js from: $(go env GOROOT)/lib/wasm/wasm_exec.js (or misc/wasm/ for Go <1.24)" 925117/b047/importcfg ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/1/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 925117/b242/_pkg_.a V7o_/18xeupG6XnJInX8DV7o_ ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE t/internal/tag GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE ntio/asm/base64 GOMODCACHE 64/pkg/tool/linux_amd64/vet env 925117/b255/_pkg_.a GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE b/gh-aw/pkg/giturev-parse GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12345/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD 925117/b013/syma--show-toplevel ieAM-Lz/1rKZ6KO_c5WKOia6VGY3 env 925117/b241/_pkg_.a GO111MODULE 925117/b241=> GOINSECURE t/internal/langurev-parse GOMODCACHE /opt/hostedtoolcache/go/1.25.8/x64/pkg/tool/linu-tests (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE g/x/net/dns/dnsmrev-parse 925117/b012/syma--show-toplevel 64/pkg/tool/linux_amd64/vet ache�� 925117/b256/_pkg_.a jfLv/0caWgwAWMGdke8fejfLv 64/pkg/tool/linux_amd64/compile GOINSECURE v3 GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/12346/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env mpiledOutput196956734/001 n.go 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name rg/x/text@v0.36.0/feature/plural/common.go 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/drbg GOMODCACHE 64/pkg/tool/linux_amd64/vet env 4002497977 NNuM/NZNs7zEf3uyY_7BzNNuM .cfg GOINSECURE t/message GOMODCACHE /Microsoft.CPlat.Core.RunCommandLinux-1.0.9/bin/initial commit (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE /go-yaml/ast 925117/b047/syma--show-toplevel 64/pkg/tool/linux_amd64/vet env fq9m/dOvw0liy9-ZbDdQ2fq9m GO111MODULE eutil.test GOINSECURE GOMOD GOMODCACHE eutil.test (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/2/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE entropy GOMODCACHE 64/pkg/tool/linux_amd64/vet env 4002497977 h5RJ/fhSiz4P0ozPJ9_2Hh5RJ 925117/b238=> GOINSECURE t/feature/pluralinit GOMODCACHE /opt/hostedtoolcache/go/1.25.8/x/tmp/file-tracker-test2138297381/test2.lock.yml (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 447096304/.github/workflows K4mu/EjXw-bZiDcnvohS5K4mu ck GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/3/artifacts --jq .artifacts[].name mLsRemoteWithRealGitbranch_with_hyphen1422370452/001' 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name om/santhosh-tekuri/jsonschema/v6@v6.0.2/kind/kin-ifaceassert x_amd64/link GOINSECURE fips140only GOMODCACHE x_amd64/link env 925117/b235/_pkg_.a wyMD/ZnqvKWWFy1YdeRMpwyMD 64/pkg/tool/linux_amd64/compile GOINSECURE osh-tekuri/jsonsrev-parse GOMODCACHE Pe/6w_qA2Or0sjRkYGevO-i/tgf3Rt9AY-ftGYeOPjLO (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 .cfg 64/pkg/tool/linu-importcfg GOINSECURE /go-yaml/scannerrev-parse GOMODCACHE 64/pkg/tool/linu/home/REDACTED/work/gh-aw/gh-aw/pkg/timeutil/format_test.go env 447096304/.github/workflows GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-trimpath (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/4/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE fips140/aes/gcm GOMODCACHE 64/pkg/tool/linux_amd64/vet env 4002497977 ho52/RILG8Ja3npv64jHUho52 ache/go/1.25.8/x64/pkg/tool/linu-nilfunc GOINSECURE osh-tekuri/jsonsrev-parse GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-tests (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE /go-yaml/internarev-parse ache/go/1.25.8/x--show-toplevel 64/pkg/tool/linux_amd64/vet env hwOz/e9PA8pRtJ2At5smDhwOz GO111MODULE ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ache/go/1.25.8/x64/pkg/tool/linu-importcfg (http block)
    • Triggering command: /usr/bin/gh gh api --paginate repos/{owner}/{repo}/actions/runs/5/artifacts --jq .artifacts[].name GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path -c=4 -nolocalimports -importcfg /tmp/go-build2355114303/b414/importcfg -pack /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/fileutil.go /home/REDACTED/work/gh-aw/gh-aw/pkg/fileutil/tar.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD abi/abi_test.s go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 xcontext GOMODCACHE 64/pkg/tool/linugit-upload-pack '/tmp/TestParseDefaultBranchFromLsRemoteWithRealGitmaster_branch1806576938/001' env -json GO111MODULE x_amd64/vet sm); \ wasm-optgit GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE 64/pkg/tool/linu/home/REDACTED/work/gh-aw/gh-aw/pkg/styles/theme.go env rity1059650655/001 .cfg 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/compile (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v0.47.4
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 1835377053/001 5114303/b031/vetrev-parse ache/go/1.25.8/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet /usr/bin/git se 5114303/b122/vet-lh .cfg git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v0.47.4 --jq [.object.sha, .object.type] | @tsv --show-toplevel go /usr/bin/git 1913033355/.githgit GO111MODULE ache/go/1.25.8/x--show-toplevel git rev-�� v1.0.0 go /usr/bin/git -json GO111MODULE ache/go/1.25.8/x/tmp/gh-aw/aw-feature-branch.patch git (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv md l_test.go 64/pkg/tool/linux_amd64/compile GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linuremote.origin.url cat-�� g_.a k16k/fLMkIGmGY3YvZBL1k16k ache/go/1.25.8/x64/pkg/tool/linux_amd64/vet GOSUMDB GOWORK 64/bin/go ache/go/1.25.8/x64/pkg/tool/linuInitial commit (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq [.object.sha, .object.type] | @tsv y_with_explicit_repo1287661141/0remote.origin.url GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env e-analyzer.md GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq [.object.sha, .object.type] | @tsv che/go-build/ab/-p GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc--revs -o /tmp/go-build232--thin -trimpath 64/bin/go -p main -lang=go1.25 go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env _.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv adata/main.go GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq [.object.sha, .object.type] | @tsv cut -f1))" GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv -json GO111MODULE Linux-1.0.9/bin/-buildmode=exe GOINSECURE GOMOD GOMODCACHE Linux-1.0.9/bin/initial commit env _.a GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq [.object.sha, .object.type] | @tsv k/gh-aw/gh-aw/cm-errorsas GOPROXY 64/bin/go GOSUMDB GOWORK 64/bin/go /opt/hostedtoolc--jq -o /tmp/go-build2322961917/b388/_pkGOINSECURE -trimpath 64/bin/go -p main -lang=go1.25 go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv 925117/b212/_pkg_.a t2Bi/LbyKJAzlPTfrrG8ct2Bi x_amd64/vet GOINSECURE nal/fips140tls GOMODCACHE x_amd64/vet -V=f�� 925117/b058/importcfg bdZx/dTcSRunIDvppKmvAbdZx .cfg GOSUMDB GOWORK 64/bin/go ache/go/1.25.8/x64/pkg/tool/linutest-branch (http block)
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq [.object.sha, .object.type] | @tsv y_with_repos=public_479861386/001 GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE k GOINSECURE GOMOD GOMODCACHE 4c65a837 (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE fips140/rsa GOMODCACHE 64/pkg/tool/linux_amd64/vet (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.8/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build2355114303/b400/cli.test /tmp/go-build2355114303/b400/cli.test -test.testlogfile=/tmp/go-build2355114303/b400/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE x_amd64/compile -w -t oken.go x_amd64/asm OUTPUT -d 168.63.129.16 x_amd64/asm (http block)
    • Triggering command: /tmp/go-build2085264560/b400/cli.test /tmp/go-build2085264560/b400/cli.test -test.testlogfile=/tmp/go-build2085264560/b400/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD erignore go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name -t security x_amd64/compile OUTPUT -d 168.63.129.16 x_amd64/compile env -json GO111MODULE x_amd64/link GOINSECURE GOMOD GOMODCACHE x_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name 2961917/b387/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/cmd/gh-aenv GOMODCACHE go env YB3w/GP3ChI12UDCGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 2961917/b387/imp-buildtags (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI and others added 3 commits April 16, 2026 04:56
This reverts commit 98ee784.

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix empty COPILOT_MODEL fallback in byok-copilot feature Fix BYOK Copilot model fallback to avoid empty COPILOT_MODEL in compiled workflows Apr 16, 2026
Copilot AI requested a review from lpcox April 16, 2026 05:27
@github-actions
Copy link
Copy Markdown
Contributor

Hey @Copilot 👋 — great work on fixing the BYOK Copilot model fallback! Ensuring that BYOK compilations always emit a non-empty COPILOT_MODEL is exactly the kind of targeted, well-scoped fix this project needs.

This PR looks ready for maintainer review. 🎉

Generated by Contribution Check · ● 1.8M ·

@github-actions github-actions bot added the lgtm label Apr 16, 2026
@lpcox lpcox marked this pull request as ready for review April 16, 2026 13:35
Copilot AI review requested due to automatic review settings April 16, 2026 13:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes byok-copilot workflow compilation so COPILOT_MODEL never falls back to an empty string when a model isn’t explicitly configured, avoiding Copilot CLI BYOK failures that require an explicit model.

Changes:

  • Add a BYOK-specific non-empty fallback for COPILOT_MODEL when features.byok-copilot is enabled and no model is configured.
  • Introduce a shared constant for the BYOK fallback model (CopilotBYOKDefaultModel).
  • Extend the existing fallback-env rendering test to cover BYOK vs non-BYOK behavior.
Show a summary per file
File Description
pkg/workflow/copilot_engine_execution.go Adjusts Copilot execution env generation to use a non-empty COPILOT_MODEL fallback when byok-copilot is enabled.
pkg/constants/engine_constants.go Adds CopilotBYOKDefaultModel constant used for BYOK fallback rendering.
pkg/workflow/model_env_vars_test.go Extends coverage to assert BYOK-specific fallback rendering for COPILOT_MODEL.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 3/3 changed files
  • Comments generated: 0

@github-actions github-actions bot mentioned this pull request Apr 16, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 100/100

Excellent test quality

Metric Value
New/modified tests analyzed 3 table-driven scenarios in 1 test function
✅ Design tests (behavioral contracts) 3 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 3 (100%)
Duplicate test clusters 0
Test inflation detected No (15 test lines / 9 prod lines ≈ 1.67:1)
🚨 Coding-guideline violations None

Test Classification Details

Test File Classification Issues Detected
TestCopilotFallbackModelMapsToNativeEnvVar — "Agent job maps GH_AW_MODEL_AGENT_COPILOT to COPILOT_MODEL" (updated) pkg/workflow/model_env_vars_test.go:218 ✅ Design None — correctly pins expectedTail: "''" to guard against regression
TestCopilotFallbackModelMapsToNativeEnvVar — "Agent job with byok-copilot uses non-empty COPILOT_MODEL fallback" (new) pkg/workflow/model_env_vars_test.go:227 ✅ Design None — tests the key behavioral contract of the fix
TestCopilotFallbackModelMapsToNativeEnvVar — "Detection job maps GH_AW_MODEL_DETECTION_COPILOT to COPILOT_MODEL" (updated) pkg/workflow/model_env_vars_test.go:236 ✅ Design None — correctly pins expectedTail: "''"

Analysis

The PR modifies a single table-driven test function by:

  1. Adding a new table row for the byok-copilot feature flag path, asserting that the compiled workflow contains COPILOT_MODEL: $\{\{ vars.GH_AW_MODEL_AGENT_COPILOT || 'claude-sonnet-4.6' }} — directly verifying the behavioral contract of the fix.
  2. Updating two existing rows to pin expectedTail: "''", ensuring that non-BYOK paths continue to generate an empty fallback and don't silently regress.

All assertions use t.Errorf with descriptive messages that include the full expected string and the actual content — easy to diagnose on failure. The test interacts with the real workflow compiler (no mocks), consistent with project guidelines. Build tag //go:build !integration is present on line 1. ✅


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 3 scenarios (unit, //go:build !integration)
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 (no JS tests changed)

Verdict

Check passed. 0% of new/modified test scenarios are implementation tests (threshold: 30%). No coding-guideline violations detected.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

References: §24513311519

🧪 Test quality analysis by Test Quality Sentinel · ● 439.9K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 100/100. Test quality is excellent — 0% of new/modified test scenarios are implementation tests (threshold: 30%). No coding-guideline violations detected.

@lpcox lpcox merged commit 0cc16d3 into main Apr 16, 2026
43 of 45 checks passed
@lpcox lpcox deleted the copilot/fix-empty-copilot-model-fallback branch April 16, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: byok-copilot feature generates empty COPILOT_MODEL fallback, breaking BYOK mode

3 participants