Retry dual-era backend startup on a lost port-bind race - #6080
Merged
Conversation
jhrozek
requested review from
ChrisJBurns,
JAORMX,
amirejaz,
aponcedeleonch,
rdimitrov and
reyortiz3
as code owners
July 28, 2026 07:55
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6080 +/- ##
==========================================
- Coverage 72.26% 72.24% -0.03%
==========================================
Files 728 728
Lines 75523 75523
==========================================
- Hits 54580 54561 -19
- Misses 17040 17057 +17
- Partials 3903 3905 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JAORMX
approved these changes
Jul 28, 2026
3 tasks
allocateVMCPPort probes for a free port and closes its own listener immediately, but the actual bind happens later, asynchronously, inside the container Docker starts for the detached `thv run` process. Under back-to-back workload churn another container can grab the same ephemeral port first, failing with "address already in use" and leaving the spec's BeforeEach to time out after 2 minutes. Retry once with a freshly allocated port when this specific failure is observed. Confirmed live under a 20-repeat stress run: the race still fires (~1 in 12-13 attempts), but the retry now absorbs it instead of failing the spec. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
jhrozek
force-pushed
the
fix-dual-era-port-bind-race
branch
from
July 28, 2026 09:12
5838b25 to
1a0802c
Compare
JAORMX
approved these changes
Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test/e2e/vmcp_dual_era_test.go'sBeforeEachintermittently failswaiting for a yardstick backend to become
running, with the containererroring out as
driver failed programming external connectivity ... bind: address already in use.allocateVMCPPort(vmcp_cli_helpers_test.go)probes for a free port and closes its own listener immediately, but the
actual bind happens later, asynchronously, inside the container Docker
starts for the detached
thv runprocess — under back-to-back workloadchurn, another container can grab the same ephemeral port first. This is a
classic check-then-use race across process boundaries; the only way to
make progress in the face of it is to retry, not to check earlier/later.
startYardstickLegacyOnPort/startYardstickModernOnPortnow gothrough a shared
startEraBackendOnPorthelper that retries once with afreshly allocated port if
WaitForMCPServerfails with"address already in use".Type of change
Test plan
task test-e2e)Ran
ginkgo --label-filter='vmcp && dual-era' --repeat=20 -vlocally againstthis fix. All 20 repeats (140 specs) passed. The port race fired once during
the run (visible in the log as
lost a port-bind race on <port>, retrying with a fresh port) and was absorbed by the retry instead of failing thespec — confirming both that the race is real and that the fix works.
API Compatibility
v1beta1API, OR theapi-break-allowedlabel is applied and the migration guidance is described above.Does this introduce a user-facing change?
No. Test-only change.
Special notes for reviewers
This fix is scoped to the two call sites that actually race Docker's async
container bind.
allocateVMCPPortis also used elsewhere forthv vmcp serve's own port, which is bound synchronously in-process by the same testbinary and isn't subject to this race, so I left
allocateVMCPPortitselfunchanged rather than touching its other callers.
Generated with Claude Code