Skip to content

[Visual Test] Label preflight rejects duplicate label, skipping Lavapipe screenshot capture #950

Description

@MichaelFisher1997

Workflow

https://github.com/OpenStaticFish/ZigCraft/actions/runs/30148576016

Failure output

label with name "run-visual-test" already exists; use `--force` to update its color and description
##[error]Process completed with exit code 1.

This is the full stderr emitted by the "Ensure visual-test label exists" step at 07:05:36.999Z in the failed run (build-output.log is never produced — the zig build run invocation is skipped because the workflow uses if: success() on the subsequent steps). The failure pattern is identical to runs 29677776782, 29725595085, 29809988559, 29899854913, 29987782599, 30074923197, and the visual-test workflow has been failing every day with this exact error since at least 2026-07-12.

Diagnosis

The visual-test workflow does not fail in the Vulkan / screenshot capture path. The Zig build and Lavapipe screenshot run never execute because the preflight gh label list / gh label create step exits non‑zero, which causes every later step (including the Run menu screenshot capture step that contains the actual zig build run -Dscreenshot-path=screenshot.png -Dskip-present=true invocation) to be skipped via if: success(). The "Failure output" above is therefore not a build-log message — build-output.log does not exist in the workspace because the build was skipped.

The actual root cause is in .github/workflows/visual-test.yml:55-68 ("Ensure visual-test label exists"):

if ! gh label list --json name --jq '.[].name' | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75"
fi
if ! gh label list --json name --jq '.[].name' | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75"
fi

gh label list paginates (default 30 per page) and --jq '.[].name'" only emits the **first page** of label names. The grep -q '^run-visual-test$'therefore returns 1 (no match) wheneverrun-visual-testis not on the first page, and the workflow then tries to recreate it. If the label *is* present (e.g., on a later page),gh label createerrors withlabel with name "run-visual-test" already exists; use --force to update its color and description. Either branch causes set -e` to abort the step with exit code 1.

The diagnostic prompt /.github/prompts/visual-test-diagnose.md misdirects the model toward a Vulkan / PPM failure that never runs — the "build-output.log" file referenced in that prompt is intentionally never created by the run-with-log action because the screenshot capture step is skipped before it executes.

Origin of the failure

  • File: .github/workflows/visual-test.yml
  • Step: Ensure visual-test label exists (lines 55-68)
  • Function: gh label list --json name --jq '.[].name' — paginated output, only first 30 labels returned
  • Logic: if ! ... grep -q '^run-visual-test$'; then gh label create ...; fi — recreates an already-existing label

Suggested fix

Either fetch all pages when listing labels, or skip create errors that already mean the label exists:

if ! gh label list --json name --jq '.[].name' --paginate | grep -q '^visual-test$'; then
  gh label create "visual-test" \
    --description "Issues from automated visual regression tests" \
    --color "E06C75" || true
fi
if ! gh label list --json name --jq '.[].name' --paginate | grep -q '^run-visual-test$'; then
  gh label create "run-visual-test" \
    --description "Run deterministic visual regression workflow on a PR" \
    --color "E06C75" || true
fi

Adding --paginate ensures run-visual-test is found on later pages, and || true on the create calls makes the step idempotent against transient gh label create failures (network/API errors, partial outages observed in earlier runner logs with Failed to restore: Cache service responded with 400 / Our services aren't available right now). The actual screenshot capture path (Lavapipe, headless swapchain at modules/engine-graphics/src/vulkan_swapchain.zig:127-174, PNG writer at modules/engine-graphics/src/vulkan/screenshot.zig:279-349, frame counting at src/game/app.zig:528-593) should then be exercised end-to-end and reported on its own merits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghotfixvisual-testIssues from automated visual regression tests

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions