[Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30431891169

[MichaelFisher1997]

Workflow

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30431891169

Failure output

The workspace post-run contained only weston.log; both build-output.log and screenshot.png were absent, indicating the screenshot step never executed. The visual-test step outputs read:

screenshot_exists=false
exists=false   # build-output.log check

No screenshot path error, no Vulkan/device/swapchain output, no Zig compile output, no Lavapipe ICD resolution output were captured, because the workflow short-circuited before the Setup Lavapipe Vulkan and Run menu screenshot capture steps ran.

The visual-test-diagnose.md prompt's hypothesis (screenshot.ppm rejected by screenshot.zig:detectScreenshotFormat) is stale for this run: the workflow on dev already uses -Dscreenshot-path=screenshot.png (.github/workflows/visual-test.yml:81), and the game itself was never invoked to test that path.

Diagnosis

Same root cause as #956 (run #30338232720), which PR #957 already diagnoses: .github/workflows/visual-test.yml:55-68:

- name: Ensure visual-test label exists
  run: |
    if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
      gh label create "visual-test" \
        --description "Issues from automated visual regression tests" \
        --color "E06C75"
    fi
    if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
      gh label create "run-visual-test" \
        --description "Run deterministic visual regression workflow on a PR" \
        --color "E06C75"
    fi
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

The repository now has 41 labels. gh label list paginates at 30 entries per page by default, so both visual-test and run-visual-test (which sit on page 2 — see https://github.com/OpenStaticFish/ZigCraft/labels?page=2) are not returned by the unflagged gh label list --json name invocation. The guard if ! ... | grep -q '^visual-test$' therefore evaluates true and the script attempts to create both labels. The existing labels cause gh label create to fail with:

label with name "visual-test" already exists; use --force to update its color and description
label with name "run-visual-test" already exists; use --force to update its color and description

exit 1 propagates from the step. Because .github/workflows/visual-test.yml:73-87 (Run menu screenshot capture) has no if: always(), it is skipped, and so are Check screenshot exists (which then sees screenshot_exists=false) and Check build log exists (exists=false). Only the Start headless Wayland compositor step that ran earlier produces the artifact (weston.log).

Result: zero diagnostic information about Zig/Vulkan/Lavapipe/SDL is reachable; the Zig/Vulkan/screenshot code path is never exercised. This explains why build-output.log is missing in the workspace even though run-with-log would have created it had the step run.

This is the second consecutive scheduled visual-test run to hit this bug; the fix from #957 is still open against dev and has not landed, so today's schedule run reproduced the same failure mode.

Suggested fix

Either page through gh label list (e.g. gh label list --json name --paginate --jq '.[].name', or pass --limit 100) or skip creation when the underlying gh label create would fail (drop the guard entirely and rely on gh label create || true, or pass --force to overwrite the existing color/description). The minimal patch is:

- name: Ensure visual-test label exists
  run: |
    set -euo pipefail
    existing=$(gh label list --json name --paginate --jq '.[].name')
    echo "$existing" | grep -q '^visual-test$' || gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75"
    echo "$existing" | grep -q '^run-visual-test$' || gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75"
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

Additionally, the Run menu screenshot capture step (.github/workflows/visual-test.yml:73-87) should be marked if: always() and the screenshot / build-log existence checks should not depend on its outcome, so that a transient preflight failure still surfaces a useful build log instead of a silent no-op.

Origin

• Failing workflow step: Ensure visual-test label exists at .github/workflows/visual-test.yml:55-68
• Failing API call: gh label list --json name --jq '.[].name' (default 30-page pagination)
• Failing guard: if ! gh label list ... | grep -q '^visual-test$' followed by unconditional gh label create
• Downstream consequence: .github/workflows/visual-test.yml:73-87 (Run menu screenshot capture) skipped
• Related: [Visual Test] Label preflight aborts before screenshot capture #945, [Visual Test] Label preflight skips Lavapipe screenshot capture #946, [Visual Test] Workflow fails before game runs: 'Ensure visual-test label exists' step rejected by duplicate label #947, [Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950, [Visual Test] Label preflight rejects duplicate label on run #30192579914, skipping Lavapipe screenshot capture #952, [Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954, [Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956 (all the same preflight regression)
• Related fix in progress: PR Filed issue #956: label preflight skips game run. #957

[github-actions]

Analysis

Confirmed against .github/workflows/visual-test.yml:55-68. The diagnosis in this issue is correct and matches #956 (run #30338232720) and the pending PR #957 — same root cause, same fix shape, same unmerged state.

Root cause (verified)

- name: Ensure visual-test label exists
  run: |
    if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
      gh label create "visual-test" ...
    fi
    ...
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

• gh label list --json name defaults to 30 entries per page. With 41 labels in the repo, both visual-test and run-visual-test sit on page 2 and are not in the response.
• The if ! ... | grep -q '^visual-test$' guard therefore evaluates true, so the script invokes gh label create "visual-test" and gh label create "run-visual-test".
• Both creates fail with label with name "..." already exists; use --force to update its color and description, returning exit 1.
• The step has no continue-on-error: true, so every subsequent step is skipped. Setup Lavapipe Vulkan, Run menu screenshot capture (line 73-87), Check screenshot exists (line 89-97), and Check build log exists (line 117-120) never run, which matches the observed screenshot_exists=false / exists=false outputs and the missing build-output.log artifact.
• The only artifact left behind is weston.log from Start headless Wayland compositor (line 32-33), which already ran.

The diagnosis prompt's screenshot.ppm hypothesis is stale — the workflow already passes -Dscreenshot-path=screenshot.png (line 81), but the Zig/Vulkan/screenshot path is never exercised because the preflight short-circuits the job first.

Suggested fix (minimal, drop-in)

Page through labels so the existence guard works regardless of label count, and stop calling gh label create once the label is known to exist:

- name: Ensure visual-test label exists
  run: |
    set -euo pipefail
    existing=$(gh label list --json name --paginate --jq '.[].name')
    echo "$existing" | grep -q '^visual-test$' || gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75"
    echo "$existing" | grep -q '^run-visual-test$' || gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75"
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

--paginate is the cleanest variant (works at any label count without hardcoding a limit). --limit 100 is a reasonable alternative if you want to keep the response shape simple.

On the secondary if: always() suggestion

Marking Run menu screenshot capture (line 73-87) if: always() would not actually help here — that step is gated by the upstream Ensure visual-test label exists step failing, not by its own outcome. The preflight step itself needs to stop propagating failure (|| true around the gh label create call, or --force to overwrite the existing color/description) so that downstream steps run regardless.

The more useful defensive change would be to make Setup Lavapipe Vulkan and Run menu screenshot capture robust to preflight failures so a future regression in the label step doesn't silently hide Zig/Vulkan/screenshot failures. Concretely: either add continue-on-error: true to Ensure visual-test label exists or split label bootstrap out of the visual-test job into a one-off workflow that is independent of the screenshot path.

Scope

This is a single bug with one root cause (label-list pagination in the preflight step) and a tightly scoped fix (one YAML block). No need to break it into sub-issues.

Tracking

• Duplicate of [Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956 (same failure mode, same line numbers)
• Fix proposed in PR Filed issue #956: label preflight skips game run. #957 (still open against dev); the suggested patch above is compatible with that PR and can be folded into it
• Related preflight regressions: [Visual Test] Label preflight aborts before screenshot capture #945, [Visual Test] Label preflight skips Lavapipe screenshot capture #946, [Visual Test] Workflow fails before game runs: 'Ensure visual-test label exists' step rejected by duplicate label #947, [Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950, [Visual Test] Label preflight rejects duplicate label on run #30192579914, skipping Lavapipe screenshot capture #952, [Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954

Recommend landing PR #957 (or this drop-in variant) before the next scheduled run at 43 4 * * * to break the consecutive failure streak.

[github-actions]

Comment posted on issue #958: #958 (comment)

Summary of analysis:

• Classification: Bug (CI/workflow preflight failure)
• Validation: Diagnosis confirmed by reading .github/workflows/visual-test.yml:55-68. The Ensure visual-test label exists step has no continue-on-error, gh label list returns only page 1 (30 of 41 labels), so both visual-test and run-visual-test (on page 2) appear missing, the guard fires, and gh label create fails with "already exists" → exit 1 skips everything downstream including Run menu screenshot capture.
• Scope: Single bug, one root cause, one YAML-block fix — no need to split.
• Duplicate: Same root cause as [Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956; fix already proposed in PR Filed issue #956: label preflight skips game run. #957 (still open). The drop-in patch with --paginate is compatible with that PR.
• Note on secondary suggestion: Adding if: always() to the screenshot step wouldn't help here — the upstream label step is what fails outright. A more useful defensive change is to either continue-on-error the label step or add || true / --force so label bootstrap failures can't hide screenshot regressions.

opencode session  |  github run