Skip to content

ci: add smoke test after building release binaries#161

Merged
adnaan merged 2 commits intomainfrom
feat/release-smoke-tests
Feb 22, 2026
Merged

ci: add smoke test after building release binaries#161
adnaan merged 2 commits intomainfrom
feat/release-smoke-tests

Conversation

@adnaan
Copy link
Copy Markdown
Contributor

@adnaan adnaan commented Feb 22, 2026

Summary

  • Adds a smoke test step to the CLI release workflow between "Build binary" and "Create archive"
  • Verifies the binary executes (./tinkerdown version), reports the correct ldflags-injected version, and handles help
  • Runs natively on each platform's build runner (macos-13, macos-14, ubuntu-latest)

Closes #128

Test plan

  • Review the workflow diff — single step insertion, no other changes
  • Verify on next release tag push or manual workflow_dispatch

🤖 Generated with Claude Code

Verifies each platform binary executes, reports the correct version
(ldflags injected), and handles the help subcommand before archiving.

Closes #128

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings February 22, 2026 18:26
@claude
Copy link
Copy Markdown

claude Bot commented Feb 22, 2026

PR Review

This is a clean, minimal addition that directly addresses #128. The placement (between build and archive) is correct, and since the matrix runners match their target OS/arch, native execution works for all three platforms. Good work overall.

One issue worth fixing

Regex metacharacter risk in grep -q

Version strings like 1.0.0 contain literal dots, but grep treats . as "any character." This means version 1.0.0 would accidentally match 1X0Y0 in the binary output. Use -F (fixed-string) to match literally:

- name: Smoke test
  run: |
    ./tinkerdown version
    ./tinkerdown version | grep -Fq "${{ steps.version.outputs.version }}"
    ./tinkerdown help

This is especially relevant if pre-release suffixes like 1.0.0-rc.1 are ever used (the - and . in the suffix are also regex metacharacters).

Minor suggestion

./tinkerdown version is invoked twice -- once for visible log output and once piped to grep. You can avoid the double execution by capturing the output:

- name: Smoke test
  run: |
    VERSION_OUT=$(./tinkerdown version)
    echo "$VERSION_OUT"
    echo "$VERSION_OUT" | grep -Fq "${{ steps.version.outputs.version }}"
    ./tinkerdown help

This keeps the log output visible while only running the binary once for the version check.

Looks good

  • Step ordering is correct (after build, before archive)
  • All three runners (macos-13/amd64, macos-14/arm64, ubuntu/amd64) build for their native arch, so no cross-execution issues
  • ./tinkerdown help as a basic sanity check is reasonable
  • No unintended side effects on surrounding steps

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a lightweight smoke test to the CLI release workflow to validate the freshly built tinkerdown binary runs correctly (and reports the expected ldflags-injected version) before packaging and publishing release artifacts.

Changes:

  • Insert a “Smoke test” step after the build step to run ./tinkerdown version and ./tinkerdown help
  • Add a version-string check against ${{ steps.version.outputs.version }} before creating the release archive

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/cli-release.yml Outdated
- name: Smoke test
run: |
./tinkerdown version
./tinkerdown version | grep -q "${{ steps.version.outputs.version }}"
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The version check uses grep without fixed-string mode, so dots in semver (e.g. 1.2.3) are treated as regex wildcards and could match an incorrect version output. Use fixed-string matching (e.g., grep -F) and ideally anchor to the full expected output to make this smoke test reliable.

Suggested change
./tinkerdown version | grep -q "${{ steps.version.outputs.version }}"
./tinkerdown version | grep -Fq "${{ steps.version.outputs.version }}"

Copilot uses AI. Check for mistakes.
Comment thread .github/workflows/cli-release.yml Outdated
Comment on lines +73 to +74
./tinkerdown version
./tinkerdown version | grep -q "${{ steps.version.outputs.version }}"
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./tinkerdown version is executed twice in this smoke test. Consider capturing the output once and reusing it for both printing and verification to keep the workflow step simpler and reduce redundant execution.

Suggested change
./tinkerdown version
./tinkerdown version | grep -q "${{ steps.version.outputs.version }}"
VERSION_OUTPUT="$(./tinkerdown version)"
echo "$VERSION_OUTPUT"
echo "$VERSION_OUTPUT" | grep -q "${{ steps.version.outputs.version }}"

Copilot uses AI. Check for mistakes.
Address review feedback: use grep -F so semver dots aren't treated as
regex wildcards, and capture version output once instead of running the
binary twice.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@claude
Copy link
Copy Markdown

claude Bot commented Feb 22, 2026

Review

The smoke test step looks good. The current implementation already addresses both common pitfalls:

  1. grep -Fq (fixed-string) — correct. Version strings like 1.0.0 contain dots that are regex metacharacters; -F ensures a literal match.
  2. Single binary execution with captured output — correct. VERSION_OUT=$(./tinkerdown version) runs the binary once and reuses the output for both logging and the grep check.

One small nit

./tinkerdown help exits 0 on most CLI frameworks, but if the binary ever returns a non-zero exit code for help (e.g. flag parsing errors), this step would fail and block the release. Worth keeping an eye on, but not a blocker.

Overall

Minimal, well-placed, and correctly implemented. Good to merge once the test plan items are verified on the next tag push.

@adnaan adnaan merged commit 2407082 into main Feb 22, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FOLLOW-UP: Add smoke tests after building release binaries

2 participants