Repository navigation
[Bug] dsh web process crashes with uncaught ENOENT when a subprocess spill temp dir disappears
#5249
yuhanwanli99-cpu
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Summary The dsh web server process dies abruptly (whole-process crash) when the subprocess output-spill temp directory is removed before a stream overflows. This is reproducible on a fresh launch.
Version
dsh/CLI: 0.1.1-rc.2 (also present in current master and v0.1.2-alpha.3)
Node: v22.23.2 (Linux, containerized host where /tmp is cleaned periodically)
Observed crash (stack trace)
node:fs:560
return binding.open(
^
Error: ENOENT: no such file or directory, open '/tmp/dsh-subprocess-TM9I5B/dsh-subprocess-67-3-9134bc97f1ff-stdout.log'
at openSync (node:fs:560:18)
at OutputCollector.spillAll (packages/subprocess/subprocess-local/lib/index.js:666:19)
at OutputCollector.push (...index.js:642:75)
at Socket. (...index.js:811:14)
at Socket.emit (node:events:519:28)
...
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/tmp/dsh-subprocess-TM9I5B/dsh-subprocess-67-3-9134bc97f1ff-stdout.log'
Root cause In packages/subprocess/subprocess-local/src/spawn.ts, the spill directory is created once per process and cached:
let defaultSpillDir: string | undefined
function privateSpillDir(): string {
defaultSpillDir ??= mkdtempSync(join(tmpdir(), 'dsh-subprocess-')) // /tmp/dsh-subprocess-TM9I5B
return defaultSpillDir
}
When a child stream overflows the in-memory cap, OutputCollector.spillAll opens a spill file with no error handling:
this.spillFile = join(this.spillDir,
dsh-subprocess-${process.pid}-${++spillCounter}-...-${this.label}.log)this.spillFd = openSync(this.spillFile, 'wx', 0o600) // throws unguarded ENOENT if the dir is gone
The call site is a bare stream.on('data', ...) handler:
stream.on('data', (chunk) => { collector.push(chunk) })
So if the cached /tmp/dsh-subprocess-XXXX directory is removed out from under a long-lived process (tmp cleaner / container /tmp reset), the openSync throws an uncaught exception that propagates out of the socket data handler and aborts the whole Node process.
I believe the key gap is that dsh installs an unhandledRejection guard (installFailLoud in packages/boot/app-boot) but no uncaughtException guard, so a synchronous throw from a native FS call in an event callback is fatal.
Impact
Any environment where /tmp/dsh-subprocess-* can be deleted while the process is alive (containers, hosts with tmp cleaners) will crash the whole dsh web server.
No in-session error is logged; the active turn is only closed as { kind: 'interrupted' } by crash recovery.
This is not an OOM, not a model/provider error, and not a native addon segfault — it is a missing error-handling path for a curated temp dir that can disappear.
Suggested fix
In OutputCollector.spillAll, wrap the openSync/writeSync in try/catch; on failure, fall back to in-memory-only collection (spillDisabled = true) rather than throwing — or recreate the spill dir.
Optionally add a global uncaughtException guard so a stray sync throw is logged instead of aborting the process.
Optionally: derive the spill dir lazily per-spill and validate it still exists, or use a per-spill mkdtemp under a directory the process owns the lifecycle of.
Minimal repro
dsh web
while running, delete /tmp/dsh-subprocess-*
trigger any subprocess (bash tool, LSP) whose output overflows maxBytes
=> whole process aborts with the ENOENT above
I can provide a patch if useful. Thank you!
All reactions