Skip to content

fix(cli): keep the published binary's stderr off the blocking write path - #15496

Merged
os-litant merged 2 commits into
mainfrom
claude/issue-14874-published-cli-blocking-stdio
Sep 4, 2026
Merged

fix(cli): keep the published binary's stderr off the blocking write path#15496
os-litant merged 2 commits into
mainfrom
claude/issue-14874-published-cli-blocking-stdio

Conversation

@os-litant

Copy link
Copy Markdown
Collaborator

Fixes #14874

The published os binary can be parked in the kernel, silently, by anything that pipes its output and stops draining. This lands the repair on the published entry point, and ships the guard that until now existed but was never packed.

The reading, on the shipped binary

node bin/run.js dev --verbose --database memory:// from a fixture app, stdout and stderr piped to a reader that stops draining, /proc/PID/fdinfo/{1,2} and /proc/PID/syscall sampled every 50 ms from outside the process:

   63 ms  fd1 O_NONBLOCK=false  fd2 O_NONBLOCK=false   before process.stdout/stderr are materialised
  112 ms  fd1 O_NONBLOCK=true   fd2 O_NONBLOCK=true    node sets it when it opens the pipes
 2829 ms  fd1 O_NONBLOCK=FALSE  fd2 O_NONBLOCK=FALSE   os dev spawns os serve --dev, stdio inherit
 2930 ms  fd1 O_NONBLOCK=true   fd2 O_NONBLOCK=true    the child materialised ITS OWN stdio on the SHARED description
 5244 ms  fd1 O_NONBLOCK=true   fd2 O_NONBLOCK=FALSE   the esbuild service, a GRANDCHILD, inherits stderr

Reader stops at 14.0 s. At 17.1 s, 4 of 4 runs:

PARK DETECTED  syscall=1(write)  fd=2  O_NONBLOCK=false  state=S
               wchan=sock_alloc_send_pskb
               cmd: node .../packages/cli/bin/run.js serve --dev --verbose   (MAIN thread)

Parked 28.9 s. SIGINT ignored while parked — alive and still in write five seconds later, 3 of 3. Released only when the reader resumed, at which point the deferred SIGINT was processed and the stack exited. Negative control (same binary, the serve spawn changed to pipe + manual forwarding): flags stay O_NONBLOCK=true for every sample, zero parks over a 30 s stoppage, SIGINT killed the responsive child in 75-79 ms.

Not a crash and not a timeout: alive, idle, unresponsive, empty log.

Two readings that decide the shape of the fix:

  1. The clearing is reversible. Any node child that materialises its own stdio re-sets O_NONBLOCK on the shared description ~100 ms later — which is why fd 1 escapes and fd 2 does not. fd 2 is left blocking for the whole run, at every log level.
  2. The durable clearing is a GRANDCHILD. os dev to os serve --dev (inherited stdio) to the esbuild service (inherited stderr), landing on the description shared all the way up. ⇒ No change to this CLI's own spawn sites could have prevented it, so the re-assert has to sit on the write path.

Severity calibration, both directions: at the default log level the same two clearings occur and fd 2 is blocking for the whole run, but no park fired in 90 s — an idle dev server emits ~0.5 KB/s and never fills the 64 KiB pipe, against ~30 KB/s at --verbose. The hazardous state is unconditional; the park needs a burst of output while the reader is away — a rebuild, a request log, an error dump, or --verbose, which is what CI usually runs.

Why this was a packaging bug as much as a wiring one

The repair already existed, was correct, and was pinned — at packages/cli/bin/stderr-nonblocking.mjs. It shipped to nobody.

packages/cli/package.json  files: ["dist", "README.md", "CHANGELOG.md"]      (no bin/ anywhere)
                           bin:   { objectstack: "./bin/run.js", os: "./bin/run.js" }
npm pack --dry-run         bin/run.js is the ONLY packed file under bin/

npm packs a bin target regardless of files, which is why bin/run.js reached every published install and the module beside it reached none. So "wire up the guard that already exists" was not available as written: the guard had to move somewhere the existing whitelist already admits.

What changed

  • packages/cli/bin/stderr-nonblocking.mjs moves to packages/cli/src/utils/stderr-nonblocking.ts, so tsc compiles it into dist/ and the existing files whitelist ships it. ⛔ files is not widened and scripts/check-published-files.mjs needs no new EXTRA_ENTRIES registration.
  • bin/run.js installs it before run(), through the same lazy ../dist/ import the file already documents for invocation.js: a static ../dist/ import would turn an unbuilt tree's "command not found" into a module-resolution error and break the classification every gate that shells out to this CLI depends on.
  • bin/run-dev.js imports the same module from ../src/ through tsx, so the source shim stays buildless — which its suite's whole subject requires.

Nothing about which arguments the CLI accepts, what it prints, or what it exits with changes. src/utils/format.ts's refusal of setBlocking(true) is untouched and stands: this is its inverse, and what keeps its premise true.

The pins, and why they have this shape

A pin that cannot run in CI is not a pin, and a pin that only checks the module exists is not one either. The full field reproduction is neither — it needs a fixture app, a dev server, a reader that stops at the right moment, a deliberate output burst and /proc, and it takes ~30 s a run. So it is the evidence above, and the pins are these:

  • test/published-entry-stderr-nonblocking.e2e.test.ts (new). Runs the real bin/run.js under node --import, with stderr as a pipe nobody reads. Inside that process it waits for the guard, then manufactures the identical hazard (spawnSync(node -e 0, { stdio: 'inherit' }) clears the same flag on the same shared description in ~30 ms), asserts the flag really was cleared, and writes 2 MiB at the absent reader. Green means the shipped binary installed the shipped guard and the burst returned; ~200 ms.
  • The same file also pins the packaging invariant that was false: the specifier bin/run.js imports the guard from must resolve to a path the files whitelist admits — read from the manifest, not hard-coded — with bin/run.js itself asserted not admitted as the control, because that asymmetry is what caused the defect.
  • test/run-dev-stderr-nonblocking.e2e.test.ts (existing) keeps its two manufactured arms and stays buildless: the guard is now TypeScript, so the harness type-strips it with esbuild (already a dependency) into a temp module rather than requiring a build, and bin/run-dev.js reaches the .ts source through tsx exactly as before.

Verification

Built at e75e9119b2; the gate union below was run at that commit on a clean tree.

Ablations, each with the mutation proven on disk (removed text counted to 0, an injected marker counted to 1, blob hash changed) and each restore proven by whole-tree git status --porcelain plus blob equality against HEAD:

leg mutation predicted measured
1 delete the guard install from bin/run.js the park returns syscall=1(write) fd=0x2 count=0x2000, wchan=sock_alloc_send_pskb, flags 02000002 (O_NONBLOCK false), SIGKILL at 23.0 s, WRITES RETURNED never written
1b same mutation, driven through the new pin the pin reds 5 of 5 cases red, first message bin/run.js did not install keepStderrNonBlocking()
2 make the guard a one-shot at install instead of per-write still parks GUARD INSTALLED after 0 ms and it parked anyway — same syscall, same wchan, SIGKILL at 5.2 s

Leg 2 is the one that matters for the design: the guard installed, and a spawn afterwards still cleared the flag out from under it. Per-write is load-bearing, not a preference. Leg 2 was rebuilt on both legs and both directions were checked in the artifact with node scripts/ablation-dist-preflight.mjs @objectstack/cli MARKER — present on the mutate leg (hit packages/cli/dist/utils/stderr-nonblocking.js), absent across all 480 built files on the restore leg. Leg 1 has no dist/ leg to prove: bin/run.js is outside tsconfig.build.json's rootDir and ships verbatim as the bin target.

Suites (pnpm --filter @objectstack/cli exec vitest run --maxWorkers=2): both stderr files, 10 passed (10), both classified |integration| by the tier predicate with no list to update.

Gate union, derived on a clean tree at final HEAD e75e9119b2 with node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands, plus the five artifact-roster families whose baseline sits under a directory this change touches (silence there is evidence in neither direction, so they were run). 54 of 54 green, each exit code captured before any pipe.

The two the packaging route was most likely to move both had something to say and both say it green:

  • check:published-files69 publishable package(s) ... declare a files whitelist that covers every entry point ... and admits no test, test-harness config or build script; 1 publish more than dist/ + README.md + CHANGELOG.md, each with a registered reason. The guard reaching dist/ needs no registration; keeping it in bin/ would have.
  • check:dual-build-cjs-loads — green, and its population is the reason it could not have moved: --list reports 103 require entry points across 66 packages and @objectstack/cli is in none of them (it declares no require condition at all).

Both of those first came back exit 3, PREREQUISITE NOT MET (with check:i18n-coverage) because they read built output from packages outside this change's closure. Exit 3 is NOT MEASURED, not a pass — so the prerequisite was satisfied with a full turbo run build and both were re-run to the green quoted above (check:i18n-coverage: 13 config(s), 621 baselined untranslated string(s), none new).

pnpm --filter @objectstack/cli typecheck green — including check:test-typecheck, which is what puts test/ in front of tsc here (tsconfig.json includes only src): 3 file(s) / 28 error(s) / 6 pinned signature(s), the ledger unchanged, so the two new test files arrived with zero.

Lint is a declared narrowing, not a full-repo run: eslint --no-inline-config --format json over the 7 changed lintable files, 7 accepted into the run, 0 errors, 0 warnings. The count is eslint's own (--format json), and the narrowing excludes nothing because this repo runs one eslint.config.mjs which "never enables type-aware linting (no parserOptions.project, no typed @typescript-eslint rules) for ANY file" (its own header, with the positive control it was measured against) — so no untouched file's verdict can move on this diff. CI runs the repo-wide sweep regardless.

⛔ Deliberately not touched: the eight inherited-stdio spawn sites (dev.ts:222 · :471 · :583 · :683, environments/bind.ts:84, init.ts:873, start.ts:241 · :444) — the measurement above says changing them would not have prevented this; src/utils/format.ts; packages/cli/package.json's files; scripts/check-published-files.mjs.

Not to be confused with two neighbouring cards, neither of which this PR addresses. #14832: the same mechanism on the unpublished bin/run-dev.js with a reader that paused — repaired on that card. #14858: the same file with a reader that went away, symptom an uncaught write EPIPE and exit 1 at ~1.4 s — a different defect.


🤖 Generated with Claude Code

https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N


Generated by Claude Code

`bin/run.js` now installs `keepStderrNonBlocking()` before oclif can write a
byte, and the guard compiles from `src/` into `dist/` so a published install
actually carries it.

Measured on the built binary: `os dev --verbose` piped to a reader that stops
draining parks the main thread in `write(2)` 3.1 s later, 4 of 4 runs, fd 2,
`O_NONBLOCK=false`, `wchan=sock_alloc_send_pskb` — parked 28.9 s, ignoring
SIGINT, released only by the reader resuming. The clearing that persisted came
from a grandchild (the esbuild service, inherited stderr), so the re-assert has
to sit on the write path rather than run once at startup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01D47qPfEWVPmhguWgBZCi5N
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/cli, touching 5 documentable anchor(s). ⚠️ 2 changed file(s) yielded no anchor (packages/cli/bin/run-dev.js, packages/cli/bin/run.js), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/protocol/kernel/lifecycle.mdx (via INSTALLED (symbol, a top-level const object))
What this run could not see
  • 2 changed file(s) yielded no anchor (packages/cli/bin/run-dev.js, packages/cli/bin/run.js) — pages documenting those are invisible to this run
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 22 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 6c9f34f203b89a588b237ee7d6d3ceeec6d2d3bapackageMentionDocs.

Which tree this was computed on

This run read content/docs from 8735c09c23714bae7177a04cdaefaf6031c4260a — the merge of head e75e9119b2aa0a9c4590af43a56842f0f01a8ab9 into base 6c9f34f203b89a588b237ee7d6d3ceeec6d2d3ba, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 8735c09c23714bae7177a04cdaefaf6031c4260a && git checkout 8735c09c23714bae7177a04cdaefaf6031c4260a
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 6c9f34f203b89a588b237ee7d6d3ceeec6d2d3ba e75e9119b2aa0a9c4590af43a56842f0f01a8ab9 && git checkout -B drift-repro 6c9f34f203b89a588b237ee7d6d3ceeec6d2d3ba && git merge --no-ff e75e9119b2aa0a9c4590af43a56842f0f01a8ab9

node scripts/docs-audit/affected-docs.mjs --json 6c9f34f203b89a588b237ee7d6d3ceeec6d2d3ba

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 6c9f34f203b89a588b237ee7d6d3ceeec6d2d3ba → pass the list as
args.docs, on the commit named under Which tree this was computed on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The PUBLISHED CLI puts its own stdout/stderr on the blocking write path every time it spawns a child with inherited stdio

2 participants