Skip to content

Five @objectstack/cli e2e test files fail on macOS on a clean checkout (port-drift arms never see the drift they assert) #12884

Description

@hotlong

Observation, filed unassigned while running pnpm --filter @objectstack/cli test for #12866. Not caused by that change — the point of this card is that it is caused by nothing in the diff.

What was measured

On a macOS host (Darwin 25.5.0), pnpm --filter @objectstack/cli test reports:

Test Files  5 failed | 194 passed (199)
     Tests  3 failed | 2258 passed | 14 skipped (2275)

The same five files were then run on a CLEAN worktree detached at dc75ba857 (the merge base, with none of #12866's changes present) after building --filter '@objectstack/cli...':

Test Files  5 failed (5)
     Tests  3 failed | 28 passed | 14 skipped (45)

Identical file set, identical failing-test set. The five:

  • test/serve-port-drift-notice.e2e.test.ts
  • test/serve-port-readback.e2e.test.ts
  • test/serve-process-child-env.e2e.test.ts
  • test/login-json-ndjson.e2e.test.ts
  • test/cloud-login-json-ndjson.e2e.test.ts

The shape of the port ones

All three serve-* failures are the same assumption: the test holds a port with a neighbour socket and expects the spawned os serve child to be pushed OFF it, so the drift can be observed. On this host the child binds the held port anyway, so there is no drift and the arm inverts:

AssertionError: the child bound the port it was asked for — there is no drift to announce
  : expected 54154 not to be 54154 // Object.is equality
 ❯ test/serve-port-drift-notice.e2e.test.ts:262:13

serve-port-readback's load-bearing arm then reads promise resolved "{ stdout: '', …(1) }" instead of rejecting — same cause one layer up.

That is a platform difference in how a held listening socket is (not) contended, not a defect in the code under test, and the tests are written as if the contention is guaranteed.

Why it is worth a card

CI runs Linux, so this is presumably green there and invisible — but every agent and every contributor working this repo on a Mac gets five red files on a clean checkout, which is exactly the noise that trains people to ignore a red suite. Either the port arms need a host-independent way to hold a port (or a documented skip), or the platform gap should be recorded so the next reader does not re-diagnose it.

The two *-login-json-ndjson.e2e.test.ts files failed at file level on both trees and were not diagnosed further here.

Generated by Claude Code

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions