[Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30522565636

[MichaelFisher1997]

Workflow

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30522565636

Symptom

The workspace post-run contained only weston.log; build-output.log and screenshot.png are both absent. Because Run menu screenshot capture (line 73-87 of .github/workflows/visual-test.yml) is reached only after every prior step succeeds, no Zig/Vulkan/Lavapipe/SDL output was captured. The visible step outputs read:

screenshot_exists=false
exists=false   # build-output.log check

There is no error: ..., no Vulkan device log, no Lavapipe ICD resolution output, no screenshot path-format error — the screenshot step itself never ran.

Diagnosis

Same root cause as #945, #946, #947, #949, #950, #952, #954, #956, #958 (all reproduces of the same preflight bug, with PR #957 still proposing the fix on dev). .github/workflows/visual-test.yml:55-68 is the offending step:

- name: Ensure visual-test label exists
  run: |
    if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
      gh label create "visual-test" \
        --description "Issues from automated visual regression tests" \
        --color "E06C75"
    fi
    if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
      gh label create "run-visual-test" \
        --description "Run deterministic visual regression workflow on a PR" \
        --color "E06C75"
    fi
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

• gh label list --json name defaults to 30 entries per page. The repository now has 41+ labels, so visual-test and run-visual-test (which sit on page 2 — see https://github.com/OpenStaticFish/ZigCraft/labels?page=2) are not returned.
• The if ! ... | grep -q '^visual-test$' guard therefore evaluates true and the script tries to create both labels.
• gh label create returns label with name "..." already exists; use --force to update its color and description and exits 1.
• The step has no continue-on-error: true, so all subsequent steps are skipped: Setup Lavapipe Vulkan, Run menu screenshot capture (line 73-87), Check screenshot exists (line 89-97), Check build log exists (line 117-120). Only Start headless Wayland compositor (line 32-33) had already run, leaving weston.log as the sole artifact.

The diagnosis prompt's screenshot.ppm hypothesis is stale for this run: .github/workflows/visual-test.yml:81 already passes -Dscreenshot-path=screenshot.png, and modules/engine-graphics/src/vulkan/screenshot.zig:262-268 (detectScreenshotFormat) accepts that extension. The Zig/Vulkan/screenshot code is never exercised because the preflight short-circuits the job first.

Suggested fix (drop-in)

- name: Ensure visual-test label exists
  run: |
    set -euo pipefail
    existing=$(gh label list --json name --paginate --jq '.[].name')
    echo "$existing" | grep -q '^visual-test$' || gh label create "visual-test" \
      --description "Issues from automated visual regression tests" \
      --color "E06C75"
    echo "$existing" | grep -q '^run-visual-test$' || gh label create "run-visual-test" \
      --description "Run deterministic visual regression workflow on a PR" \
      --color "E06C75"
  env:
    GH_TOKEN: ${{ secrets.OPENCODE_PAT }}

--paginate is the robust variant (no label-count assumption); --limit 100 is equivalent for the current state. This patch is compatible with the still-open PR #957.

Defensive follow-ups worth considering

• Add continue-on-error: true (or simply || true around gh label create) to Ensure visual-test label exists so a future regression in label bootstrap cannot silently hide Zig/Vulkan/screenshot failures.
• Optionally lift label bootstrap out of visual-test.yml into a one-off bootstrap workflow so it never gates the screenshot path again.

Tracking

• Duplicates: [Visual Test] Label preflight aborts before screenshot capture #945, [Visual Test] Label preflight skips Lavapipe screenshot capture #946, [Visual Test] Workflow fails before game runs: 'Ensure visual-test label exists' step rejected by duplicate label #947, [Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950, [Visual Test] Label preflight rejects duplicate label on run #30192579914, skipping Lavapipe screenshot capture #952, [Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954, [Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956, [Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30431891169 #958.
• Fix proposed in PR Filed issue #956: label preflight skips game run. #957 (open against dev, not landed yet). Recommend merging PR Filed issue #956: label preflight skips game run. #957 (or applying this drop-in) before the next scheduled run (43 4 * * *) to break the consecutive-failure streak.

[github-actions]

Analysis Summary

Classification: Bug (CI workflow infrastructure, not Zig/Vulkan code).

Validation:

• The root cause is correctly identified: gh label list defaults to 30 entries and the repo has 41 labels (verified via gh label list --json name --limit 100 --jq 'length' → 41).
• Confirmed .github/workflows/visual-test.yml:55-68 lacks pagination.
• Confirmed .github/workflows/visual-test.yml:81 already passes -Dscreenshot-path=screenshot.png and modules/engine-graphics/src/vulkan/screenshot.zig:262-268 accepts .png, so the stale PPM hypothesis is moot.

Two corrections the issue gets wrong that should be addressed before merging:

1. --paginate does not exist on gh label list. gh label list --help shows no such flag — only -L, --limit int. The drop-in should use --limit 100 (works today) or, more robustly, loop on --limit 100 --paginate if/when GitHub CLI adds it. A gh api call against /repos/:owner/:repo/labels?per_page=100 is the most future-proof variant.
2. PR Filed issue #956: label preflight skips game run. #957 does not fix this bug. It only edits devenv.lock (flake cleanup); the workflow file is untouched. Treating it as "the fix" would let the chain continue. A real PR that edits .github/workflows/visual-test.yml:55-68 is still needed.

Duplicates confirmed (open, all visual-test): #916 (original), #931, #935, #937, #942, #944, #945, #946, #947, #949, #950, #952, #954, #956, #958, plus this #960. The cron 43 4 * * * keeps re-firing the same failure daily.

Recommended next steps (no code changes from me, per instructions):

• Open (or re-target) a PR against dev that fixes .github/workflows/visual-test.yml:55-68 using gh api repos/{owner}/{repo}/labels?per_page=100 or --limit 100, with set -euo pipefail and || true on gh label create so bootstrap can never gate the screenshot path again.
• Optionally also update .github/prompts/visual-test-diagnose.md to drop the stale .ppm hypothesis so future diagnosis prompts stop chasing a non-issue.
• Close [Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30522565636 #960 (and the duplicate chain [Visual Test] Label preflight fails: gh label list only returns first 30 labels, run-visual-test is on page 2 #916/[Visual Test] Label preflight regressed: gh label list pagination skips run-visual-test on run #19 #931/[Visual Test] Label preflight regressed again: unpaginated gh label list still misses run-visual-test on run #20 #935/[Visual Test] Label preflight skipped screenshot capture on run #21 #937/[Visual Test] Label preflight prevents screenshot capture on run #26 #942/[Visual Test] Label bootstrap failure prevents menu screenshot run #944/[Visual Test] Label preflight aborts before screenshot capture #945/[Visual Test] Label preflight skips Lavapipe screenshot capture #946/[Visual Test] Workflow fails before game runs: 'Ensure visual-test label exists' step rejected by duplicate label #947/[Visual Test] PPM screenshot path is rejected before Vulkan readback #949/[Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950/[Visual Test] Label preflight rejects duplicate label on run #30192579914, skipping Lavapipe screenshot capture #952/[Visual Test] Label preflight skips Lavapipe screenshot capture on run #30248571304 #954/[Visual Test] Label preflight rejects duplicate label on run #30338232720, skipping Lavapipe screenshot capture #956/[Visual Test] Label preflight pagination skips Lavapipe screenshot capture on run #30431891169 #958) as duplicates once the fix PR is merged and the next scheduled run is green.

opencode session  |  github run