[Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30689332702

[MichaelFisher1997]

Workflow

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30689332702

Failure output

The workspace post-run contained only weston.log; both build-output.log and screenshot.png were absent, indicating the screenshot step never executed. The visual-test step outputs read:

screenshot_exists=false
exists=false   # build-output.log check

No screenshot path error, no Vulkan/device/swapchain output, no Zig compile output, no Lavapipe ICD resolution output were captured, because the workflow short-circuited before the Setup Lavapipe Vulkan and Run menu screenshot capture steps ran.

The visual-test-diagnose.md prompt's hypothesis (screenshot.ppm rejected by screenshot.zig:detectScreenshotFormat) is stale for this run: the workflow on dev already uses -Dscreenshot-path=screenshot.png (.github/workflows/visual-test.yml:81), and modules/engine-graphics/src/vulkan/screenshot.zig:262-268 (detectScreenshotFormat) accepts that extension. The Zig/Vulkan/screenshot code is never exercised because the preflight short-circuits the job first. I verified this by reading the staged 8d205ff (the commit pinned on dev) — visual-test.yml:81 reads zig build run -Dscreenshot-path=screenshot.png -Dskip-present=true, and the only .ppm references left in the source tree are stb_image.h, the unrelated worldgen-climate-snapshot writer, and the stale visual-test-diagnose.md prompt itself.

Diagnosis

Same root cause as #916, #931, #935, #942, #944, #945, #946, #947, #949, #950, #952, #954, #956, #958, #960, #962 (reproduces of the same preflight bug, with PR #953 still proposing the fix on dev). .github/workflows/visual-test.yml:55-68 is the offending step:

- name: Ensure visual-test label exists
  run: |
    if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
      gh label create "visual-test" \
        --description "Issues from automated visual regression tests" \
        --color "E06C75"
    fi
    if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
      gh label create "run-visual-test" \
        --description "Run deterministic visual regression workflow on a PR" \
        --color "E06C75"
    fi
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

• gh label list --json name defaults to 30 entries per page. The repository now has 41 labels, so visual-test (alphabetical position 21) is still returned, but run-visual-test (alphabetical position 41) is not. I confirmed this locally:

  $ gh label list --json name --jq '.[].name' | grep -E 'visual-test|run-visual-test'
  visual-test
  $ gh label list --json name --limit 100 --jq '.[].name' | grep -E 'visual-test|run-visual-test'
  visual-test
  run-visual-test
  

• The second if ! ... | grep -q '^run-visual-test$' guard therefore evaluates true and the script tries to create the label.

• gh label create returns label with name "run-visual-test" already exists; use --force to update its color and description and exits 1.

• The step has no continue-on-error: true, so all subsequent steps are skipped: Setup Lavapipe Vulkan, Run menu screenshot capture (line 73-87), Check screenshot exists (line 89-97), Check build log exists (line 117-120). Only Start headless Wayland compositor (line 32-33) had already run, leaving weston.log as the sole artifact.

The diagnose prompt's screenshot.ppm hypothesis is stale for this run for two independent reasons:

1. .github/workflows/visual-test.yml:81 already passes -Dscreenshot-path=screenshot.png, not .ppm.
2. Even if the workflow did pass .ppm, the screenshot capture path is never reached because the label preflight exits 1 first.

Suggested fix

Two-line fix that resolves the recurring failure and stops future diagnosis agents from being misdirected by the stale prompt:

1. Make the label preflight idempotent and paginated in .github/workflows/visual-test.yml:55-68:

   - name: Ensure visual-test labels exist
     run: |
       for label in visual-test run-visual-test; do
         if ! gh label list --paginate --json name --jq '.[].name' | grep -q "^${label}$"; then
           gh label create "${label}" \
             --description "Issues from automated visual regression tests" \
             --color "E06C75" || true
         fi
       done
     env:
       GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

2. Update .github/prompts/visual-test-diagnose.md:7,42 so the prompt no longer assumes -Dscreenshot-path=screenshot.ppm (the workflow has been using .png for a while):

   - 2. Builds and runs the game with Lavapipe (software Vulkan): `zig build run -Dscreenshot-path=screenshot.ppm -Dskip-present=true`
   - 3. The game should render the HomeScreen menu for 5 frames, capture a screenshot as PPM, and exit
   - 4. The PPM is then converted to PNG
   + 2. Builds and runs the game with Lavapipe (software Vulkan): `zig build run -Dscreenshot-path=screenshot.png -Dskip-present=true`
   + 3. The game should render the HomeScreen menu for 5 frames, capture a screenshot as PNG, and exit

   - The game uses `-Dscreenshot-path=screenshot.ppm` which sets `build_options.screenshot_path`
   - This enables screenshot mode: loads HomeScreen instead of WorldScreen, counts frames, captures PPM via Vulkan, then exits
   + The game uses `-Dscreenshot-path=screenshot.png` which sets `build_options.screenshot_path`
   + This enables screenshot mode: loads HomeScreen instead of WorldScreen, counts frames, captures PNG via Vulkan, then exits

   And remove the .ppm-flavored items from the failure-pattern checklist (lines 17-23), or replace them with the actual failure that dominates the workflow today (Ensure visual-test label exists exit 1 with label with name "run-visual-test" already exists).

PR #953 already proposes the workflow fix; the only change needed for this run is merging it and removing the stale .ppm references from the diagnose prompt.

Reference

• Workflow run: https://github.com/OpenStaticFish/ZigCraft/actions/runs/30689332702
• Failure point: .github/workflows/visual-test.yml:55-68 (Ensure visual-test label exists step)
• Same root cause as: [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916, [Visual Test] Label preflight regressed: gh label list pagination skips run-visual-test on run #19 #931, [Visual Test] Label preflight regressed again: unpaginated gh label list still misses run-visual-test on run #20 #935, [Visual Test] Label preflight prevents screenshot capture on run #26 #942, [Visual Test] Label bootstrap failure prevents menu screenshot run #944, [Visual Test] Label preflight aborts before screenshot capture #945, [Visual Test] Label preflight skips Lavapipe screenshot capture #946, [Visual Test] Workflow fails before game runs: 'Ensure visual-test label exists' step rejected by duplicate label #947, [Visual Test] PPM screenshot path is rejected before Vulkan readback #949, [Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950, [Visual Test] Label preflight rejects duplicate label on run #30192579914, skipping Lavapipe screenshot capture #952, [Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954, [Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956, [Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30431891169 #958, [Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30522565636 #960
• Proposed fix PR: Label preflight paginates, skips screenshot run. #953 (open)
• Stale prompt references: .github/prompts/visual-test-diagnose.md:7,21,42,43

[github-actions]

Classification: Bug (recurring CI infrastructure failure)

Validation summary

Confirmed locally against the working tree:

• .github/workflows/visual-test.yml:55-68 is the failure point. The second if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$' guard exits non-zero when the label is past the default 30-item page, then gh label create "run-visual-test" exits 1 with label with name "run-visual-test" already exists; use --force to update its color and description, and the step has no continue-on-error, so Setup Lavapipe Vulkan (line 70), Run menu screenshot capture (line 73), the screenshot/log existence checks (lines 89, 117), and the artifact uploads are all skipped. Only weston.log survives.
• .github/workflows/visual-test.yml:81 already passes -Dscreenshot-path=screenshot.png.
• modules/engine-graphics/src/vulkan/screenshot.zig:262-268 (detectScreenshotFormat) accepts .png and returns null for anything else — so the diagnose prompt's .ppm rejection hypothesis is wrong even if the workflow did pass that path, which it never does any more.
• .github/prompts/visual-test-diagnose.md still tells the agent to expect a .ppm build/convert pipeline (lines 7-9, 21, 42-43) and never mentions the actual dominant failure (the label preflight). This is the real reason the same root cause keeps being re-filed as a new issue.

Duplicates and existing fix

• Author correctly identifies this as the same root cause as [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916, [Visual Test] Label preflight regressed: gh label list pagination skips run-visual-test on run #19 #931, [Visual Test] Label preflight regressed again: unpaginated gh label list still misses run-visual-test on run #20 #935, [Visual Test] Label preflight prevents screenshot capture on run #26 #942, [Visual Test] Label bootstrap failure prevents menu screenshot run #944-[Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950, [Visual Test] Label preflight rejects duplicate label on run #30192579914, skipping Lavapipe screenshot capture #952, [Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954, [Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956, [Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30431891169 #958, [Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30522565636 #960, [Visual Test] Golden image is completely black, causing compare_visual_golden.sh to refuse comparison #962.
• PR Label preflight paginates, skips screenshot run. #953 (open) is the proposed one-line workflow fix (add --paginate and || true to the label preflight). It is the right fix and should be merged into dev so the cron stops producing these.
• PR Label preflight pagination skips game run. #965 is the latest runner-generated report for this run; its only file change is .venv.lock formatting churn, not a code fix. PR Label preflight paginates, skips screenshot run. #953 remains the merge target.

Scope and suggested next steps

Issue is correctly scoped — single root cause, single fix. Recommend treating it as a duplicate of #916 (or whichever of the earlier siblings is kept open) once #953 lands, and not as a new sub-issue.

Two concrete actions would unblock the cron without inventing new work:

1. Merge Label preflight paginates, skips screenshot run. #953 (paginate the gh label list calls and add || true to the gh label create calls in visual-test.yml:55-68). That's the only change needed to make the workflow stop filing this issue.
2. Independently, update .github/prompts/visual-test-diagnose.md so it stops misdirecting the diagnosis agent:
   • Replace screenshot.ppm with screenshot.png at lines 7, 9, 42, 43.
   • Update the failure-pattern checklist at lines 17-23 so the first item is the actual dominant failure: Ensure visual-test label exists step exits 1 with label with name "run-visual-test" already exists because gh label list paginated at 30 entries.
   • Add a pointer to visual-test.yml:55-68 and screenshot.zig:262-268 so future agents read the actual code path before filing.

If item 2 is split into a separate bug/ or chore/ PR instead of riding along with #953, that's fine — they're independent and the prompt fix doesn't need to block the workflow fix.

opencode session  |  github run