Skip to content

ci: validate releases in the same environment as CI - #246

Merged
oratis merged 1 commit into
mainfrom
fix/release-validate-parity
Aug 8, 2026
Merged

ci: validate releases in the same environment as CI#246
oratis merged 1 commit into
mainfrom
fix/release-validate-parity

Conversation

@oratis

@oratis oratis commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Summary

The v0.3.0 tag failed. Not a regression — a latent gap in release.yml that had never been exercised, because this was the first release.yml run the repo has ever had (v0.3.0 is the only tag).

What happened

release.yml's validate job runs pnpm test on a bare ubuntu-latest. ci.yml installs bubblewrap + slirp4netns first (with two sysctl relaxations) because the sandbox tests need them.

So this test — passing on main for months, added back in 4dc5ee9, untouched by any of #237#245 — died in the release gate:

FAIL src/tools/bash.test.ts > BashTool > fails closed to deny-all-net when the network sandbox is unavailable
  Expected: "network allowlist unavailable"
  Received: "Error spawning command: spawn bwrap ENOENT"

The deny-all-net fallback genuinely spawns bwrap on Linux. No bubblewrap, no fallback, nothing to assert.

Why this is worth fixing rather than working around

A release gate weaker than CI is worse than no gate: it makes a green CI stop predicting a green release, and the only signal you get is a failed tag after you've already pushed it. That's precisely what happened here.

The alternative — making the test tolerate a missing bwrap — would have hidden the real problem, which is that the two pipelines disagree about what "validated" means.

The validate job now mirrors ci.yml's sandbox setup and its DC_SANDBOX_NET_TEST opt-in, with a comment on the release side saying the two move together.

Blast radius of the failed tag: none

Verified before re-tagging:

Check Result
GitHub Releases none created
publish-cli, build-vscode, build-mac, github-release all skipped (validate is their dependency)

Nothing was published, so moving the tag is safe.

Test plan

  • Confirmed the failure predates this work: git log --follow packages/core/src/tools/bash.test.ts → test added in 4dc5ee9, long before feat(core): give unattended runs an explicit approval policy #237
  • Confirmed ci.yml on c4d43fc passed with the same test — the difference is the environment, not the code
  • node scripts/check-docs.mjs
  • The real verification is the re-tagged release run, which I'll watch

Documentation

  • docs/RELEASING.md — the validate step description now mentions the sandbox tooling and why

Release notes label

  • release-notes:internal — CI infrastructure

Related

Blocks the v0.3.0 release (#245).

🤖 Generated with Claude Code

The v0.3.0 tag failed on a test that had been passing on main for months. Not a
regression — release.yml's validate job runs `pnpm test` on a bare
ubuntu-latest, while ci.yml installs bubblewrap and slirp4netns first. The
deny-all-net fallback test spawns `bwrap`, so it died with ENOENT before it
could assert anything.

This was the first release.yml run the repo has ever had (v0.3.0 is the only
tag), so the gap had never been exercised.

A release gate weaker than CI is worse than no gate: it makes a green CI stop
predicting a green release, and the only signal is a failed tag. The validate
job now mirrors ci.yml's sandbox setup and its DC_SANDBOX_NET_TEST opt-in, with
a comment on both sides noting they move together.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@oratis
oratis merged commit bdb186c into main Aug 8, 2026
5 checks passed
@oratis
oratis deleted the fix/release-validate-parity branch August 8, 2026 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant