Skip to content

fix(cli): reserve stdout for the --json payload across the bootSchemaStack family (#6217) - #6524

Merged
os-project-manager merged 1 commit into
mainfrom
claude/issue-6217-json-stdout-purity
Aug 8, 2026
Merged

fix(cli): reserve stdout for the --json payload across the bootSchemaStack family (#6217)#6524
os-project-manager merged 1 commit into
mainfrom
claude/issue-6217-json-stdout-purity

Conversation

@os-project-manager

Copy link
Copy Markdown
Collaborator

Fixes #6217

The defect

--json has exactly one audience — a program — and every subcommand that boots a kernel handed that program a stream it could not parse. ObjectLogger routes debug/info/warn to stdout and only error/fatal to stderr (packages/core/src/logger.ts:343), so os migrate recorded-by --json produced roughly 60 INFO lines above the payload and two shutdown lines below it, with stderr completely empty:

$ os migrate recorded-by --json 2>/dev/null | jq .
parse error: Invalid numeric literal at line 1, column 13

The consumer's only recourse was a "find the last lone { and its matching }" extractor. #4873 was forced to write one to assert its own payload (packages/cli/test/migrate-exit-code.e2e.test.ts's jsonPayload()), and that heuristic silently picks the wrong text as soon as a log line looks like JSON.

Premise verified against origin/main before implementing — reproduced exactly as filed:

--- stdout (8344 bytes) --- stderr (0 bytes)
[StandaloneStack] no compiled artifact at '…' — booting without one (run 'os compile' to build it)
2026-08-08T02:14:38.330Z INFO Registered metadata loader: filesystem (file:)
…
JSON.parse FAILED: Unexpected token 'S', "[Standalone"... is not valid JSON

Which route, and why

The issue left three routes open. Taken: route 1 — redirect to stderr.

  • Route 2 (logLevel: 'silent' under --json) was rejected on two counts. It throws the operator's diagnostics away, warnings included; and it cannot quiet the one line that never passes through the kernel logger at all — loadArtifactBundle's bare console.log, which prints [StandaloneStack] no compiled artifact … straight to stdout. Route 2 alone would leave the invariant broken on any uncompiled project.
  • Route 3 (make the logger default to stderr in packages/core) is out of scope by ruling and stays untouched.

A correction to the dispatch note worth recording: packages/cli/src/commands/plugin/build.ts:180 and packages/cli/src/utils/build-runtime.ts:143 were cited as evidence that route 2 is reachable today. Both are esbuild's logLevel: 'silent', not the kernel logger's — they are options passed to esbuild.build({ … }). So they say nothing about the kernel logger either way. Reachability was established independently instead: LoggerConfig (packages/spec/src/system/logging.zod.ts) carries level, format, file and rotation but no destination or stream knob, so a destination change cannot be expressed through config at all — which is why the redirection happens on the stream itself. os serve already does exactly this to keep its startup banner readable: it swaps process.stdout.write for the boot window and buffers what it intercepts (packages/cli/src/utils/boot-log-capture.ts, #4012). This is that same, already-proven CLI-layer seam, pointed at stderr instead of at a buffer. No packages/core change was needed.

The fix

packages/cli/src/utils/json-stdout.ts (new) owns the reservation, and the shared boot seam installs it.

  • bootSchemaStack gains a required jsonOutput: boolean. It is taken before the boot can print its first byte — createStandaloneStack announces a missing artifact before any plugin is constructed, so a reservation one statement later already arrives too late.
  • While reserved, process.stdout.write forwards to process.stderr.write, varargs and drain callback intact. Measured: this catches direct writes and console.log / console.info / console.debug, because Node resolves process.stdout and calls .write on it per record — and ObjectLogger.write() does the same.
  • The payload leaves through writeStdoutDirect, which holds the real write; emitText (and therefore emitJson) is built on it. That is the only thing that may reach stdout during a reservation, which is what makes "exactly one JSON document" structural.
  • shutdown() releases stdout after the kernel is fully down — kernel.shutdown() is itself two INFO lines, and those printed below the payload.
  • A failed boot deliberately keeps the reservation: the command's next act on that path is to emit its error payload, and a half-started kernel can still log.

Required rather than optional so a member added to this family later has to decide at compile time instead of inheriting the bug. Booting the shared stack is what makes a command a member, so this is the one call it cannot avoid.

Family verified against origin/main (the issue's list still holds; all nine declare a json flag and call bootSchemaStack): os migrate plan / apply / resume / recorded-by / summary-nulls / value-shapes / files-to-references, os migrate meta --stored, os meta resync. One clarification: os migrate meta boots only under --stored, which is why the bare form was never affected and migrate-meta.e2e.test.ts could always JSON.parse(stdout).

Human-mode runs are unchanged — verified on the real CLI: stdout byte-identical in shape, stderr still empty.

Tests

  • packages/cli/test/json-stdout-purity.e2e.test.ts (new) — the shared family expectation. The contract has one implementation face per command, so the family is discovered from source (calls bootSchemaStack AND declares a --json flag) and reconciled against the driven list. Each member then runs as a real child process, asserting (a) a bare JSON.parse(stdout) succeeds — no extraction, (b) no kernel-logger record and no [StandaloneStack] anywhere on stdout, (c) the boot diagnostics are still present on stderr. (c) is what makes a regression toward route 2 go red here too: a fix that deletes the operator's diagnostics is a different defect. A newly added family member goes red until it is driven.
    • The fixture is deliberately uncompiled, so [StandaloneStack] no compiled artifact … — the console.log that never touches the logger — stays in play as the second, independent pollution source.
  • packages/cli/test/migrate-exit-code.e2e.test.ts — the jsonPayload() heuristic extractor is deleted; it is now a bare JSON.parse(stdout). The Graceful shutdown complete receipt moved to stderr and is asserted there, with a matching negative assertion on stdout.
  • packages/cli/src/utils/json-stdout.test.ts (new) — the mechanism at unit level: forwarding, the drain callback, the payload's way out, release, nested reservation. Note the global console is not usable as evidence inside a vitest worker (vitest replaces it), so the console coverage there goes through a node:console Console bound to the process streams, and the real global-console proof is the e2e's [StandaloneStack] assertion.

Reverse verification — predicted red, and red

With the reservation disabled (bootSchemaStack ignoring jsonOutput, everything else untouched):

 Test Files  1 failed (1)
      Tests  27 failed | 1 passed (28)      # json-stdout-purity.e2e.test.ts

27 of 28 fail. The one that stays green is the member-inventory case, which asserts a set rather than behaviour — expected, and the reason it is a separate case.

     × really did the work it is reporting success for — this is not a masked exit
     × reports its duration IN THE PAYLOAD — the value that used to be the exit code
SyntaxError: Unexpected token 'S', "[Standalone"... is not valid JSON
 Test Files  1 failed (1)
      Tests  2 failed | 3 passed (5)        # migrate-exit-code.e2e.test.ts

That SyntaxError is verbatim the error recorded in the issue — the removal of jsonPayload() is what exposes it.

Green runs

pnpm --filter @objectstack/cli typecheck        →  tsc --noEmit, clean
pnpm --filter @objectstack/cli test             →  Test Files 93 passed (93) / Tests 962 passed (962)

Every gate enumerated from .github/workflows/lint.yml was run individually — pnpm lint plus all 29 check:* steps of the ESLint job, and every step of the TypeScript Type Check job (workspace build + typecheck, the spec generated-artifact gates, i18n, ratchets). All pass.


Generated by Claude Code

…maStack family (#6217)

`--json` has exactly one audience — a program — and every subcommand that
boots a kernel was handing that program a stream it could not parse.
`ObjectLogger` routes `debug`/`info`/`warn` to stdout and only `error`/`fatal`
to stderr, so `os migrate recorded-by --json` produced "~60 INFO lines +
payload + 2 shutdown lines" on stdout while stderr stayed empty. The only
recourse was a "find the last lone `{` and its matching `}`" extractor — #4873
was forced to write one to assert its own payload — and that heuristic
silently picks the wrong text as soon as a log line looks like JSON.

This takes route 1 from the issue (redirect to stderr), not route 2 (drop the
kernel to `logLevel: 'silent'`): route 2 throws the operator's diagnostics
away, warnings included, and could not quiet the one line that never goes
through the logger at all (`console.log` in `loadArtifactBundle`:
`[StandaloneStack] no compiled artifact …`). Route 3 (make the logger default
to stderr in `packages/core`) would change `os serve` / `os dev` output for
every existing user and is a maintainer call — untouched here.

The fix lands on the family's shared boot seam: `bootSchemaStack` gains a
REQUIRED `jsonOutput` option. Before the boot can print its first byte it
takes over `process.stdout.write` and forwards everything the kernel and its
plugins write to stderr — nothing is discarded — while the payload goes out
through `writeStdoutDirect` on the real stdout. `shutdown()` gives stdout back
only after the kernel is fully down, so the two shutdown lines cannot land
under the payload either. A failed boot deliberately keeps the reservation:
the command's next act on that path is to emit its error payload, and a
half-started kernel can still log. The option is required so a family member
added later has to decide at compile time instead of inheriting the bug.

All nine members covered: `os migrate plan` / `apply` / `resume` /
`recorded-by` / `summary-nulls` / `value-shapes` / `files-to-references`,
`os migrate meta --stored`, and `os meta resync`. Human-mode output is
unchanged (verified: stdout identical, stderr still empty).

Tests:
- `packages/cli/test/json-stdout-purity.e2e.test.ts` — the shared family
  expectation. Discovers the members from source (calls `bootSchemaStack` AND
  declares a `--json` flag), reconciles that set against the driven list, then
  runs each as a real child process asserting a bare `JSON.parse(stdout)`, no
  logger record anywhere on stdout, and the boot diagnostics still present on
  stderr (so a regression toward silencing goes red too). A new member that is
  not driven goes red.
- `packages/cli/test/migrate-exit-code.e2e.test.ts` — the `jsonPayload()`
  heuristic extractor #4873 wrote under duress is deleted; it is now a bare
  `JSON.parse(stdout)`.
- `packages/cli/src/utils/json-stdout.test.ts` — the mechanism at unit level.

Reverse verification (predicted red, and red): with the reservation disabled,
27 of the family e2e's 28 cases fail — the one that stays green is the
member-inventory case, which asserts a set rather than behaviour — and the
exit-code pin fails with `SyntaxError: Unexpected token 'S', "[Standalone"...`,
exactly the error recorded in the issue.

Fixes #6217

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017uFVNMmTxLpmfQYiuKM1Yx
@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 8, 2026 3:10am

Request Review

@github-actions github-actions Bot added the size/l label Aug 8, 2026
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/cli.

21 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/skills-reference.mdx (via packages/cli)
  • content/docs/api/client-sdk.mdx (via @objectstack/cli)
  • content/docs/api/data-flow.mdx (via @objectstack/cli)
  • content/docs/api/environment-routing.mdx (via @objectstack/cli)
  • content/docs/api/error-catalog.mdx (via @objectstack/cli)
  • content/docs/automation/hook-bodies.mdx (via packages/cli)
  • content/docs/deployment/backup-restore.mdx (via @objectstack/cli)
  • content/docs/deployment/cli.mdx (via @objectstack/cli)
  • content/docs/deployment/self-hosting.mdx (via @objectstack/cli)
  • content/docs/deployment/validating-metadata.mdx (via packages/cli)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/cli)
  • content/docs/kernel/runtime-services/data-service.mdx (via @objectstack/cli)
  • content/docs/kernel/runtime-services/index.mdx (via packages/cli)
  • content/docs/permissions/authentication.mdx (via @objectstack/cli)
  • content/docs/plugins/index.mdx (via @objectstack/cli)
  • content/docs/plugins/packages.mdx (via @objectstack/cli)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/cli)
  • content/docs/protocol/kernel/realtime-protocol.mdx (via @objectstack/cli)
  • content/docs/releases/implementation-status.mdx (via @objectstack/cli)
  • content/docs/releases/v16.mdx (via @objectstack/cli)
  • content/docs/releases/v17.mdx (via @objectstack/cli)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Aug 8, 2026
@os-project-manager
os-project-manager marked this pull request as ready for review August 8, 2026 04:39
@os-project-manager
os-project-manager added this pull request to the merge queue Aug 8, 2026
Merged via the queue into main with commit 2b641dd Aug 8, 2026
25 checks passed
@os-project-manager
os-project-manager deleted the claude/issue-6217-json-stdout-purity branch August 8, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

os migrate recorded-by --json 的 stdout 里混着内核 INFO 日志,payload 无法直接 JSON.parse

2 participants