ci: verify what we publish from the consumer side - #5
Merged
Conversation
added 2 commits
August 8, 2026 12:57
Nothing in this repo has ever checked the artifact a user downloads. ci.yml proves the code builds and its tests pass; release.yml's verify-assets proves the upload step believed itself, asserted from inside the run that did the uploading. Neither is evidence about the published thing. That gap shipped twice in this fleet. webrequest-hook v1.0.4 published as a zero-asset phantom: tag, Release object, nothing to download. headroom-hook's published Docker bundle could not boot the gateway, because its shipped docker/bundle/config.yaml still used config shapes busbar 1.5.3 retired, so the container exits 1 during config load and never binds a port. Both were green everywhere. Adds consumer-verify.yml, which calls the fleet's shared GetBusbar/busbar/.github/workflows/plugin-consumer-verify.yml. It checks out nothing, so a fix that is committed but never published still fails it. It asserts the Release is published rather than a still-draft, that every platform archive it owes is present and downloadable THROUGH /releases/latest/download/ (a count can never see a missing platform, only a name can), that one archive really is a plugin busbar would accept - manifest name/alias/kind/version, the sha256 binding the cdylib beside it, a real shared library, a non-empty signature - and, where a runnable bundle is published, that the image BOOTS AND SERVES. The same job is appended to release.yml as its final step, so a broken publish turns the RELEASE red rather than leaving a green release beside a red run nobody correlates. On failure the shared workflow opens or updates ONE labelled issue naming the failing check, its expected and observed values and the run URL, and closes it again when a run passes. The inputs are read off the PUBLISHED manifest rather than guessed from the crate name: this fleet genuinely disagrees with itself about naming (the store repos drop the trailing -plugin the auth repos keep, and store-valkey publishes as busbar-store-redis).
On a pull_request event `github.ref_name` is '<number>/merge', not a branch name, so plugin-ci asked GetBusbar/busbar for a branch called '5/merge' and the sibling checkout died with 'The process /usr/bin/git failed with exit code 1', naming neither the ref nor the reason. Every pull request to this repo has been red on that before running a line of the change it was testing. `github.base_ref` is the branch the PR targets and is empty on a push, so this means dev-against-dev and main-against-main as intended on both events. plugin-ci.yml also now falls back with a loud warning when a ref cannot be resolved, so this is belt and braces - but a caller should say what it means rather than lean on a fallback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
Nothing in this repo has ever checked the artifact a user downloads.
ci.ymlproves the code builds and its tests pass.release.yml'sverify-assetsproves the upload step believed itself, asserted from inside the run that did the uploading. Neither is evidence about the published thing.That gap shipped twice in this fleet:
webrequest-hookv1.0.4 published as a zero-asset phantom. Tag, Release object, nothing to download.headroom-hook's published Docker bundle could not boot the gateway. Its shippeddocker/bundle/config.yamlstill uses config shapes busbar 1.5.3 retired (auth.admin_auth:with inline module entries), so the container exits 1 during config load and never binds a port. Verified against the current published image,getbusbar/busbar-headroom:2.0.4, while writing this.Both were green everywhere. Nothing anywhere noticed.
What it adds
.github/workflows/consumer-verify.yml, which calls the fleet's shared reusable workflowGetBusbar/busbar/.github/workflows/plugin-consumer-verify.yml@dev- one file for all ten repos, the same way every repo already callsplugin-ci.yml, so a fix reaches the whole fleet instead of being copied ten times and drifting nine.It checks out nothing, so a fix that is committed but never published still fails it. Per release it asserts:
/releases/latestresolves to it;/releases/latest/download/. Not a count - a count can never see a missing platform, only a name can;name/alias/kind/version, thesha256actually binding the cdylib beside it, the packed file really being a shared library, and a non-empty signature (every release workflow here silently falls back to--allow-unsignedwhenBUSBAR_SIGN_KEYis unset, and busbar then refuses the plugin on the user's machine);The same job is appended to
release.ymlas its final step, so a broken publish turns the release red rather than leaving a green release beside a red run nobody correlates.Consumer verification is post-publication by nature - you cannot download an asset that was never uploaded - so it cannot block the publish and does not pretend to. The value is the verdict being impossible to miss: red on the release, and an auto-filed issue. On failure the shared workflow opens or updates one labelled issue naming the failing check, its expected and observed values, and the run URL, and closes it again when a run passes.
Note on inputs
They are read off the published manifest, not guessed from the crate name. This fleet genuinely disagrees with itself about naming: the store repos drop the trailing
-pluginthat the auth repos keep, andstore-valkeypublishes asbusbar-store-redis.Merge order
Depends on GetBusbar/busbar#54, which adds the shared reusable workflow. Merge that to
busbar'sdevfirst; until then this workflow'suses:cannot resolve.