Skip to content

Smoke-test release packages from one matrix job - #29

Merged
calvintvu merged 1 commit into
mainfrom
release-smoke-matrix
Aug 5, 2026
Merged

Smoke-test release packages from one matrix job#29
calvintvu merged 1 commit into
mainfrom
release-smoke-matrix

Conversation

@calvintvu

@calvintvu calvintvu commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Context

release.yml carried the same 28-line smoke block twice — once inline in build for arm64, once in smoke-intel for amd64 — differing only in the archive stem. Two copies on the path that publishes binaries is where drift costs the most.

Both also reimplemented, less thoroughly, what collector/scripts/smoke.sh already does for make smoke.

Changes

  • Replace both blocks with one smoke matrix job over {arm64, macos-15} and {amd64, macos-15-intel}. It unpacks the archive, checks --version against the tag, and calls scripts/smoke.sh.
  • publish now needs [build, smoke].

Coverage goes up, not down. The old blocks checked only that / served aroot div and that /api/sessions parsed as JSON. smoke.sh additionally asserts the page title, that a /assets/*.js referenced by the document resolves, that /coslash survives a direct hit, that a missing asset returns 404 rather than falling back to the SPA document, and that an unrouted /api path returns 404. It also drops the third JSON validator — the old blocks used python3 -m json.tool, smoke.sh uses jq with a grep fallback.

Two things reviewers should weigh:

  • The smoke job needs actions/checkout to reach the script. That is one extra step per arch, and it means the smoke job now tests the script at the tag being released rather than a copy frozen into the workflow.
  • The arm64 run moves out of build, so artifacts upload before either arch is verified. publish needs both jobs, so nothing untested reaches a release; the artifacts are internal, with 1-day retention.

fail-fast: false so one arch failing still reports the other.

Test

  • ruby -ryaml parse — valid; jobs resolve to build → smoke → publish →bump-homebrew, matrix expands to the two expected pairs, no smoke-intel references left.
  • make -C collector dist VERSION=v9.9.9-test — passed, two archives plus checksums.txt.
  • Replayed the new job body verbatim on darwin-arm64 against the real packaged tarball: checksums verified, - version returned v9.9.9-test, and scripts/smoke.sh passed all seven assertions.
  • Confirmed neither check is vacuous: the version guard exits 1 on a mismatch, and scripts/smoke.sh <bare-binary> embedded fails with no frontend assets are embedded in this binary. That is the one failure mode that could otherwise ship behind a green check.

Not covered: the amd64 leg on real Intel hardware, and the tap-publish job — both unchanged by this PR and both unavailable locally.

The arm64 and amd64 smoke blocks were 28 lines of identical inline bash,
differing only in the archive stem. Both also reimplemented, less
thoroughly, what scripts/smoke.sh already does for `make smoke`.

Replace both with one matrix job over {arm64, macos-15} and
{amd64, macos-15-intel} that unpacks the archive, checks --version
against the tag, and calls scripts/smoke.sh. Two copies of the release
gate become one, and the release path now exercises the same script a
developer runs locally.

Coverage goes up rather than down. The old blocks checked only that /
served a root div and /api/sessions parsed as JSON. smoke.sh also
asserts the page title, that a referenced /assets/*.js resolves, that
/coslash survives a direct hit, that a missing asset 404s instead of
falling back to the SPA document, and that an unrouted /api path 404s.

The smoke job needs a checkout to reach the script, and the arm64 run
moves out of `build`, so artifacts now upload before either arch is
verified. publish needs [build, smoke], so nothing publishes untested.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@calvintvu calvintvu added the github_actions Pull requests that update GitHub Actions code label Aug 5, 2026
@calvintvu
calvintvu merged commit 0353fc6 into main Aug 5, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants