feat(ci): self-test json-schema-validation against a fixture chart

[piontec]

Part of giantswarm/giantswarm#37347 — Lack of schema testing in github-workflows lets bugs pass

Why

verify-helm-schema was bumped to v2.6.2 in #258 and JSON schema validation broke in every cluster app repo (example).

The cause was not the install-binary-action v3.0.0 → v4.1.0 bump in the same commit — that action's default download_url is unchanged, and unarchived binaries have been supported since v3.1.0.

It was schemalint's release assets changing under a download_url that still asked for the old name. The <repo>-v<version>-<platform>.tar.gz assets are built by this repo, in create-release.yaml's create_and_upload_build_artifacts job, which only runs when the caller passes build-release-artifacts: true. schemalint#319 (chore: align files according to platform standards) removed that input, so from v2.6.2 on the tarball was simply never built — while action.yml was still requesting it via install-binary-action's default URL → Unexpected HTTP response: 404.

It was live org-wide about a minute after merge, because this repo is consumed at @main. schemalint v2.6.3 fixed it with an explicit download_url and #261 pinned it, so the outage window was 14:39–16:05 UTC on 2026-08-04.

Nothing here could have caught it. Validate GitHub workflows runs yamllint, action-validator and zizmor — all static, none of which ever executes a pinned action.

What

A check that calls the real reusable workflow against a fixture chart, so the pins under test are exactly the ones that ship — there is no second copy of the pins to drift out of sync.

• helm/selftest-cluster-app/values.schema.json — schemalint's own pkg/lint/rulesets/testdata/cluster_azure.json (its rulesets_test.go asserts zero errors under cluster-app), passed through schemalint normalize, which verify also checks.
• helm/selftest-cluster-app/values.yaml — exact stdout of helm-values-gen, so the generate job's diff passes.
• The fixture has to live at the repo root: both composite actions locate input with find ./helm -maxdepth 2 -name values.schema.json, and a workflow_call job cannot be given a working directory.
• renovate.json5 — ignorePaths: ['helm/**'] so the helm-values manager leaves the fixture alone.

The paths filter means upstream rule-set churn can only redden PRs touching this workflow or the fixture, never unrelated ones — and a renovate pin bump does touch json-schema-validation.yaml, which is exactly the case that broke.

Drive-by: both actions/checkout steps now set persist-credentials: false per the repo's action rules. Neither job uses git.

Verification

Locally, with schemalint 2.6.3 and helm-values-gen 1.0.4:

• schemalint verify helm/selftest-cluster-app/values.schema.json --rule-set cluster-app → valid JSON Schema, normalized, valid against cluster-app (the ~90 recommendations are advisory and do not fail).
• diff helm/selftest-cluster-app/values.yaml <(helm-values-gen …) → empty.
• Repo yamllint → no new errors, only the pre-existing truthy warning that every workflow here already has.

A negative test is pushed to this branch and then reverted, to show the check actually fails when the pin is broken.

🤖 Generated with Claude Code

[piontec]

Negative test done, as promised in the description.

Pinned verify-helm-schema back to the broken v2.6.2 SHA (c2280a5a) in commit 43848fd. The self-test went red in 7s with the identical failure cluster-aks hit:

download_url: https://github.com/giantswarm/${binary}/releases/download/v${version}/${binary}-v${version}-linux-amd64.tar.gz
download URL: https://github.com/giantswarm/schemalint/releases/download/v2.6.2/schemalint-v2.6.2-linux-amd64.tar.gz
##[error]Unexpected HTTP response: 404

Run 31001492199 — reverted in 3beba60, and all checks are green again on the current head.

Two things came out of the investigation behind this PR:

• feat(ci): self-test the verify-helm-schema action schemalint#338 — the mirror-image guard upstream. It runs the action on release: published, which is earlier than this check: assets are final at that moment, before Renovate can open a PR here. Verified installing v2.6.3/schemalint-linux-amd64.
• fix(ci): install gitsemver from its raw release binary #263 — the same bug, armed, in this repo. gitsemver is pinned to gitsemver-v${version}-linux-amd64.tar.gz in 4 places across create-release.yaml and create-release-pr.yaml. It resolves today only because the pinned v2.0.1 predates chore: align files according to platform standards gitsemver#260 removing build-release-artifacts: true. Its next release would break every release workflow in the org.

Also worth correcting the record: my initial reading blamed the install-binary-action v3→v4 bump in the same schemalint commit. That was wrong — v4.1.0's default download_url is unchanged, and extension-less URLs have worked since v3.1.0. The real cause is the align-files template dropping build-release-artifacts: true, which is what actually builds those tarballs (in create-release.yaml, right here). Written up in giantswarm/install-binary-action#353.