ci: validate releases in the same environment as CI - #246
Merged
Conversation
The v0.3.0 tag failed on a test that had been passing on main for months. Not a regression — release.yml's validate job runs `pnpm test` on a bare ubuntu-latest, while ci.yml installs bubblewrap and slirp4netns first. The deny-all-net fallback test spawns `bwrap`, so it died with ENOENT before it could assert anything. This was the first release.yml run the repo has ever had (v0.3.0 is the only tag), so the gap had never been exercised. A release gate weaker than CI is worse than no gate: it makes a green CI stop predicting a green release, and the only signal is a failed tag. The validate job now mirrors ci.yml's sandbox setup and its DC_SANDBOX_NET_TEST opt-in, with a comment on both sides noting they move together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
v0.3.0tag failed. Not a regression — a latent gap inrelease.ymlthat had never been exercised, because this was the firstrelease.ymlrun the repo has ever had (v0.3.0is the only tag).What happened
release.yml's validate job runspnpm teston a bareubuntu-latest.ci.ymlinstallsbubblewrap+slirp4netnsfirst (with two sysctl relaxations) because the sandbox tests need them.So this test — passing on main for months, added back in
4dc5ee9, untouched by any of #237–#245 — died in the release gate:The deny-all-net fallback genuinely spawns
bwrapon Linux. No bubblewrap, no fallback, nothing to assert.Why this is worth fixing rather than working around
A release gate weaker than CI is worse than no gate: it makes a green CI stop predicting a green release, and the only signal you get is a failed tag after you've already pushed it. That's precisely what happened here.
The alternative — making the test tolerate a missing
bwrap— would have hidden the real problem, which is that the two pipelines disagree about what "validated" means.The validate job now mirrors
ci.yml's sandbox setup and itsDC_SANDBOX_NET_TESTopt-in, with a comment on the release side saying the two move together.Blast radius of the failed tag: none
Verified before re-tagging:
publish-cli,build-vscode,build-mac,github-releaseNothing was published, so moving the tag is safe.
Test plan
git log --follow packages/core/src/tools/bash.test.ts→ test added in4dc5ee9, long before feat(core): give unattended runs an explicit approval policy #237ci.ymlonc4d43fcpassed with the same test — the difference is the environment, not the codenode scripts/check-docs.mjsDocumentation
docs/RELEASING.md— the validate step description now mentions the sandbox tooling and whyRelease notes label
release-notes:internal— CI infrastructureRelated
Blocks the v0.3.0 release (#245).
🤖 Generated with Claude Code