Windows: host process crashes with ENOENT in OutputCollector.spillAll after a temp cleaner removes %TEMP%\dsh-subprocess-*

[3067997259-design]

Reporting a host-crash bug here since issues are disabled. Environment: @deepseek-ai/dsh 0.1.0-rc.5 installed globally from npm, Windows 10/11. Trigger: a %TEMP% cleaning tool running during an active session (Huorong / 火绒 in the original case; CCleaner and Windows Storage Sense are the same class).

Crash

Error: ENOENT: no such file or directory, open 'C:\Users\<user>\AppData\Local\Temp\dsh-subprocess-XDlWVx\dsh-subprocess-21012-1-fa3ac74027e2-stdout.log'
    at openSync (node:fs:560:18)
    at OutputCollector.spillAll (.../@deepseek-ai/dsh-subprocess-local/lib/index.js:417:19)
    at OutputCollector.push (.../lib/index.js:393:75)
    at Socket.<anonymous> (.../lib/index.js:562:14)
  errno: -4058, code: 'ENOENT', syscall: 'open'

The whole dsh host process dies mid-task.

Root cause

As read from packages/subprocess/subprocess-local/src/spawn.ts at rc.5:

1. privateSpillDir() creates the private spill directory with mkdtempSync once per process and caches it for the process lifetime — it is never revalidated.
2. The directory is touched again only when a command's output first overflows the in-memory collect cap. If an external cleaner removed it in between, the first openSync(file, 'wx') fails with ENOENT — the file name is freshly random, so ENOENT can only mean the parent directory is gone.
3. That throw happens synchronously inside the socket 'data' listener (stream.on('data', chunk => collector.push(chunk))), so it becomes an uncaught exception and kills the host. The -1- in the file name above is the spill counter: this was the first spill of that process, i.e. the directory sat unused through a long session and was cleaned in between.

Every neighboring path already degrades gracefully — a spill past its cap is discarded, a failed final close withholds the spill path, and dsh-spill-policy logs a failed saveText and keeps the inline result. This open/write was the one unguarded surface.

Reproduction

Start a session, delete %TEMP%\dsh-subprocess-* (or run a temp cleaner), then run any command whose collected output exceeds the in-memory cap. The host crashes on the first spill open.

Suggested fix

Two small changes in spawn.ts:

• Wrap the spill open/write in spillAll() in a catch that permanently disables spilling for that stream via the existing discardSpill() state — the failure never propagates out of push(), and the truncated tail keeps serving reads.
• Revalidate the cached directory in privateSpillDir() with existsSync on each spawn and recreate it with mkdtempSync when missing, so one deletion costs only the collectors racing it instead of every later spawn's spill.

Sibling-pattern audit: dsh-spill-local's privateRoot() is not affected — saveTextFile() recreates the session directory via recursive mkdir on every save, and its async failures are already contained by dsh-spill-policy.

---

中文小结：Windows 临时文件清理工具（火绒等）删除 %TEMP%\dsh-subprocess-* 后，会话中第一条输出超过内存上限的命令会让整个 dsh 宿主进程以未捕获异常崩溃；根因与最小修复如上，两处改动都在 dsh-subprocess-local 的 spawn.ts。

[sjh9714]

Confirming this still reproduces at 0.1.0-rc.6, not just rc.5. The shipped lib/index.js there still has both halves of it.

function privateSpillDir() {
	defaultSpillDir ??= mkdtempSync(join(tmpdir(), "dsh-subprocess-"));
	return defaultSpillDir;
}

spillAll(chunk) {
	...
	if (this.spillFd === void 0) {
		this.spillFile = join(this.spillDir, `dsh-subprocess-${process.pid}-${++spillCounter}-...`);
		this.spillFd = openSync(this.spillFile, "wx", 384);
		for (const prior of this.chunks) writeSync(this.spillFd, prior);
	}
	writeSync(this.spillFd, chunk);
}

The ??= caches for the process lifetime and the open is unguarded, so your read of the mechanism matches what is in the package.

Two details from having reimplemented this same collector for dsh-win32, in case they save someone time.

The failure really is confined to the first open after deletion, which corroborates your -1- observation. Deleting the directory does not invalidate an already-open descriptor, so a collector that had spilled before the cleaner ran keeps writing happily into the unlinked file. That is why this needs a long quiet session to show up, and it is also why a repro that spills first and deletes second will look like the bug is not there. The repro has to delete before the first overflow of that process.

The throw site is what makes it fatal rather than annoying. push is invoked straight from stream.on('data', ...), so there is no frame between it and the event loop that could catch. Every other spill failure in the file degrades, as you noted, and this one is the single unguarded surface, so the severity gap is entirely about where it is called from rather than about how likely it is.

On your suggested fix, agreed on both parts, with one adjustment to the second. existsSync and then recreate leaves a window where the cleaner deletes between the check and the open, which is exactly the interleaving that produced this report in the first place. Attempting the open and recreating on ENOENT with one retry closes that window without a stat on every spawn, and it composes with your first suggestion since the retry failing just falls into the same permanent-disable path.

For what it is worth, dsh-win32 guards this surface, so users on the Windows plugin do not hit this crash on the collect path. That is not a claim of foresight. We rewrite collect-mode stdio to raw pipes to sniff and decode legacy code pages, and we recreate the spill directory on every open rather than caching it, which happened to make the cleaner harmless. It is unit-covered now, one test for the cleaner case and one for a spill target that cannot be created at all.

One honest difference in ours. We use a fixed per-pid directory name rather than mkdtemp, which gives up the unpredictable-path property your comment in privateSpillDir is protecting. We accepted that because the plugin is win32 only and %TEMP% is per-user there, and because the open uses wx, so a pre-created file fails closed into no-spill rather than into a leak. If the core takes the recreate approach, keeping mkdtemp and re-running it on the retry is strictly better than what we do.

[3067997259-design]

Thanks — adopted, with your adjustment replacing our original second half entirely. The implementation (in our local fork; happy to hand over the patch) now does exactly what you describe: attempt the open, and on ENOENT obtain a fresh mkdtempSync directory — which also updates the process cache, so later spawns resolve the healthy dir — and retry exactly once. Any remaining failure falls into the same permanent per-stream disable, and a non-ENOENT open error (EACCES, etc.) does not trigger a refresh at all. The existsSync-per-spawn idea is dropped: as you said, its check-to-open gap is the same interleaving that produced this report, and the retry closes the window at the failure point with no stat on the happy path.

Your repro nuance is now encoded in the tests as well: a cleaner-removes-dir case healing through the refresh with exactly one retry, a refreshed-dir-also-gone case disabling spilling, a non-ENOENT open case disabling without a refresh, and a spawn-level run that deletes the default dir between two overflowing commands and asserts the second spill lands in a fresh private dir. The write-up also calls out that a repro which spills first and deletes second cannot show the bug — deletion must precede the process's first overflow, which is exactly what the -1- counter in the original stack pointed to.

On the fixed per-pid name in dsh-win32: the trade reads as sound at plugin scope (win32-only, per-user %TEMP%, 'wx' failing closed into no-spill). For the core we kept mkdtemp per the path-prediction comment and refresh with a fresh random dir on retry — same conclusion as your last line.

[zoahdev]

This is the same unguarded spillAll open I triaged earlier as #1961 — a branch already exists and ships the exact behavior you and @sjh9714 converged on.

• Branch: zoahdev/deepseek-harness@fix/subprocess-spill-recreate-on-enoent (cherry-pick-ready against 47f943859b)
• packages/subprocess/subprocess-local/src/spawn.ts: spillAll catches ENOENT on the openSync(wx), recreates the private spill directory, retries once; any remaining failure falls into the existing per-stream discardSpill() so the throw never escapes the socket data callback. A non-ENOENT open error (EACCES, etc.) still propagates unchanged.
• Regression in spawn.spec.ts; the package's Linux CI lane runs it.

One delta from your final design: I recreate the cached mkdtemp dir in place (mkdirSync recursive) rather than minting a fresh mkdtempSync on the retry. Both close the crash; the fresh-dir variant also updates the process cache so later spawns resolve the healthy dir. If maintainers prefer the fresh-dir refresh (your -1- counter and the check-to-open TOCTOU argument are both right), it's a two-line swap on top of this branch — happy to prepare that variant.

Your "delete must precede the first overflow" repro nuance is exactly why this needs a long quiet session to show: the -1- in the stack is the first spill of the process. Good write-up.