Skip to content

Comments

Fix smoke-copilot golden file to include --browser-arg --no-sandbox in Playwright entrypointArgs#17819

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-playwright-argument-issue
Feb 23, 2026
Merged

Fix smoke-copilot golden file to include --browser-arg --no-sandbox in Playwright entrypointArgs#17819
pelikhan merged 2 commits intomainfrom
copilot/fix-playwright-argument-issue

Conversation

Copy link
Contributor

Copilot AI commented Feb 23, 2026

TestWasmGolden_CompileFixtures/smoke-copilot was failing because the golden file wasn't updated when mcp_config_playwright_renderer.go added --browser-arg --no-sandbox to Playwright's entrypointArgs (to disable Chromium's sandbox for localhost access).

Changes

  • pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-copilot.golden: updated Playwright MCP entrypointArgs to match the renderer output:
    - "entrypointArgs": ["--output-dir", "/tmp/gh-aw/mcp-logs/playwright"],
    + "entrypointArgs": ["--output-dir", "/tmp/gh-aw/mcp-logs/playwright", "--browser-arg", "--no-sandbox"],

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw -w -buildmode=exe git rev-�� --show-toplevel -extld=gcc /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha sAOsG8WsF GO111MODULE 1714565/b124/vet.cfg GOINSECURE GOMOD GOMODCACHE go ortc�� -json stmain.go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linu--limit GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env 0629-27502/test-3076948618/.githgo1.25.0 GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha bot-detection.md GO111MODULE ortcfg.link GOINSECURE GOMOD GOMODCACHE 2GrHKbdPFsnY6xQzyS/oFy7E6D7PTBexQgP2YYC/SYVWI1Pjb7L_MWnrvwk8 env -json GO111MODULE g_.a GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env runs/20260223-030629-27502/test-1700823334/.github/workflows GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha GOMODCACHE sh /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet ./cmd/gh-aw GOPROXY 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet -uns�� -unreachable=false /tmp/go-build3491714565/b108/vet.cfg /usr/bin/git --check scripts/**/*.js 64/bin/go git (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha sistency_GoAndJavaScript320083354/001/test-frontmatter-with-env-template-expressions.md -trimpath clusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle -p main -lang=go1.25 git rev-�� --show-toplevel -dwarf=false /usr/bin/git go1.25.0 -c=4 -nolocalimports git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel sh /usr/bin/git "prettier" --chegit /bin/sh 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go /usr/bin/git (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE erignore env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE sh (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x-test.short=true GOINSECURE GOMOD GOMODCACHE go env runs/20260223-030629-27502/test-1278394432/.github/workflows GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go l GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha bot-detection.md GO111MODULE 64/pkg/tool/linux_amd64/link GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/link env ipts.test GO111MODULE ortcfg.link l GOMOD GOMODCACHE U8IGZ3xsiiK_VzgSGT/jqXiqKS2mm7rG7KRlb0f/T36gUPQZXFjOiSQ6baL5 (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go estl�� -json GO111MODULE 64/bin/go Action pins syngit GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env ty-test.md GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path g/cli/access_logGOINSECURE g/cli/actionlintGOMOD 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 6079423/b389/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/envu+x GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env 0abu/f0McwWvAsgAGOSUMDB GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE 6079423/b402/importcfg (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha matter-with-arra-test.timeout=10m0s GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 90104830/.github/workflows GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 6079423/b393/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/logg-c GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 6079423/b396/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE b/gh-aw/pkg/math-c GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build3491714565/b381/cli.test /tmp/go-build3491714565/b381/cli.test -test.testlogfile=/tmp/go-build3491714565/b381/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE ache/go/1.25.0/xGO111MODULE env 6079423/b357/_pkGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] CI Failure Investigation - Run #37460</issue_title>
<issue_description>### Summary
test job fails because TestWasmGolden_CompileFixtures/smoke-copilot now generates a playwright job step with --browser-arg --no-sandbox, but the golden file still expects the previous argument list.

Failure Details

Root Cause Analysis

The new Playwright change to "disable Chromium sandbox in Playwright MCP to allow localhost access" (commit 6740e50e5e21c749d84e8bce670cea08d99f80c8) adds "--browser-arg", "--no-sandbox" to the entrypointArgs for the smoke workflows. TestWasmGolden_CompileFixtures/smoke-copilot compares the compiled workflow to pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-copilot.golden, which still lists only "--output-dir", "/tmp/gh-aw/mcp-logs/playwright", so the golden diff fails as soon as this test runs.

Failed Jobs and Errors

  • test: pkg/workflow - TestWasmGolden_CompileFixtures/smoke-copilot (golden diff showing missing --browser-arg --no-sandbox in entrypointArgs)
Investigation Findings
  • Golden diff excerpt:
    --- Expected
    +++ Actual
    @@ -497,3 +497,3 @@
                    "args": ["--init", "--network", "host", "--security-opt", "seccomp=unconfined", "--ipc=host"],
    -                "entrypointArgs": ["--output-dir", "/tmp/gh-aw/mcp-logs/playwright"],
    +                "entrypointArgs": ["--output-dir", "/tmp/gh-aw/mcp-logs/playwright", "--browser-arg", "--no-sandbox"],
                    "mounts": ["/tmp/gh-aw/mcp-logs:/tmp/gh-aw/mcp-logs:rw"]
    
  • This failure is deterministic once the new Playwright call is compiled, so the test runs fail every CI run until the golden file is updated.
  • Reproduction:
    1. go test ./pkg/workflow -run TestWasmGolden_CompileFixtures/smoke-copilot -count=1
    2. Observe the diff against pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-copilot.golden.
    3. Update the golden to match the compiled workflow (or adjust the compilation logic if another change is expected).

Recommended Actions

  • Update pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/smoke-copilot.golden to include the --browser-arg --no-sandbox entries in entrypointArgs.
  • Re-run go test ./pkg/workflow -run TestWasmGolden_CompileFixtures -count=1 (or make test-unit if broader coverage is desired) to ensure no other golden mismatches surfaced.
  • Document the sandbox flag change in the changelog or release notes since it affects workflow manifests that are validated by golden tests.

Prevention Strategies

  • Whenever Playwright/MCP invocation arguments change, rerun TestWasmGolden_CompileFixtures to refresh the golden fixtures before pushing, ensuring the compiled workflow assets stay in sync with test expectations.
  • Consider adding a script that regenerates the wasm golden files automatically whenever smoke workflow templates are modified, so CI failures highlight unexpected test regressions instead of expected updates.

AI Team Self-Improvement

When you modify Playwright entrypoint arguments (e.g., to disable the sandbox), immediately rerun the wasm golden fixture tests and update pkg/workflow/testdata/wasm_golden/TestWasmGolden_CompileFixtures/*.golden files before creating CI-affecting commits.

Historical Context There are no open `[CI Failure Doctor]` issues describing this golden mismatch; the closest search hit was issue #17486, but it is closed and unrelated.

🩺 Diagnosis provided by CI Failure Doctor

To install this workflow, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. View source at https://github.com/githubnext/agentics/tree/ea350161ad5dcc9624cf510f134c6a9e39a6f94d/workflows/ci-doctor.md.

  • expires on Feb 24, 2026, 2:59 AM UTC

Comments on the Issue (you are @copilot in this section)


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

…intArgs

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix Playwright argument mismatch in CI test Fix smoke-copilot golden file to include --browser-arg --no-sandbox in Playwright entrypointArgs Feb 23, 2026
@pelikhan pelikhan marked this pull request as ready for review February 23, 2026 03:12
Copilot AI review requested due to automatic review settings February 23, 2026 03:12
@pelikhan pelikhan merged commit f5097e3 into main Feb 23, 2026
2 checks passed
@pelikhan pelikhan deleted the copilot/fix-playwright-argument-issue branch February 23, 2026 03:12
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the golden test file for the smoke-copilot workflow to match the current Playwright MCP configuration, which now includes --browser-arg --no-sandbox in the entrypoint arguments. This change was previously made to the renderer to disable Chromium's sandbox for localhost access, but the corresponding golden file was not updated, causing test failures.

Changes:

  • Updated smoke-copilot.golden to include --browser-arg --no-sandbox in Playwright's entrypointArgs

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] CI Failure Investigation - Run #37460

2 participants