[Bug] OutputCollector.spillAll writeSync is uncaught: EDQUOT/ENOSPC kills the whole dsh web host

[NOirBRight]

Environment

• Version: dsh-v0.1.2-alpha.1-cd5ef8148158 (official alpha.1 staging build)
• OS: Linux, systemd 259 (tmp.mount has x-systemd.graceful-option=usrquota)
• Host: dsh web under a user systemd unit with Restart=always
• Node: v22.22.2

中文摘要

bash 输出超过内存上限（默认 64KB）后，OutputCollector.spillAll 用同步 writeSync 往 os.tmpdir()（通常是 /tmp）写溢出文件。Linux + systemd ≥258 给 /tmp tmpfs 加了约 80% 的每用户配额。配额一满，df 仍显示有空闲，但 writeSync 抛 EDQUOT（Node 报 code: 'UNKNOWN', errno: -122）。这条写在 stream.on('data') 回调里、没有 try/catch，整个 dsh web 进程退出。systemd 拉起后，所有进行中的 turn 变成 turn/end { kind: interrupted }，看起来像模型自己停了。

同文件的 closeSync / unlinkSync 已经接住了；openSync / writeSync 没有。这不是用户点停止，也不是 model picker。

Symptom

The host Node process aborts. systemd restarts it a few seconds later. Every in-flight turn is recorded as:

{"type":"turn/end","data":{"reason":{"kind":"interrupted"}}}

Not a user abort, not a model-picker side effect. The UI just looks like the agent stopped mid-run.

Repro class

1. Linux with systemd ≥258 so /tmp is tmpfs with per-user quota (default ~80% of the tmpfs, itself ~50% of RAM).
2. Fill that quota with anything under /tmp (test stores, Playwright artifacts, leftover dsh-subprocess-* spill dirs). df /tmp can still show gigabytes free — that free space is not available to the quota-exhausted user.
3. Run any bash tool whose stdout/stderr exceeds maxOutputBytes (default 64KB). OutputCollector opens a spill file and writeSyncs.

Once (2) is true, (3) kills the whole host. After a restart the next overflowing bash does it again.

A 1-byte write to /tmp as that user fails with [Errno 122] Disk quota exceeded while df -h /tmp still reports free space.

Production evidence (2026-08-31)

Five identical crashes on one dsh web host, 11:04 / 11:12 / 11:14 / 11:15 / 11:29 local time. Representative journal:

Error: UNKNOWN: unknown error, write
    at writeSync (node:fs:916:3)
    at OutputCollector.spillAll (file://.../packages/subprocess/subprocess-local/lib/index.js:720:3)
    at OutputCollector.push (file://.../packages/subprocess/subprocess-local/lib/index.js:693:75)
    at Socket.<anonymous> (file://.../packages/subprocess/subprocess-local/lib/index.js:862:14)
    ...
  errno: -122,
  syscall: 'write',
  code: 'UNKNOWN'

Linux 122 is EDQUOT. Node/libuv does not map it, so the JS error is UNKNOWN.

Spill files created at crash time were 0 bytes: openSync(..., 'wx') succeeded, the first writeSync failed. The first crash in that series had been up 11h44m (9.5G memory peak); the later four died on the first overflow after restart because the quota was already exhausted.

Code gap

packages/subprocess/subprocess-local/src/spawn.ts:

private spillAll(chunk: Buffer): void {
  // ...
  this.spillFd = openSync(this.spillFile, 'wx', 0o600)
  for (const prior of this.chunks) writeSync(this.spillFd, prior)
  writeSync(this.spillFd, chunk)
}

Called from:

stream.on('data', (chunk: Buffer) => { collector.push(chunk) })

discardSpill() / seal() already contain closeSync / unlinkSync (and tests mock failNextClose / failNextUnlink). openSync / writeSync in spillAll are uncaught. An exception from a 'data' listener takes down the process.

The package README already notes that completed spill files accumulate under os.tmpdir() until something external cleans them. That accumulation is how a long-running web host can walk into the quota; the host death is the missing containment.

Proposed fix (for maintainers; this is not a PR)

1. Wrap openSync / writeSync in spillAll. On failure, call existing discardSpill(): disable further spilling, keep the in-memory tail. Same outcome as a spill that exceeds maxSpillBytes or a failed final close.
2. Extend the existing vi.mock('node:fs') in packages/subprocess/subprocess-local/tests/spawn.spec.ts with failNextOpen / failNextWrite. Assert push() does not throw, spillPath is undefined, and the retained tail is still present. Cover both “open fails” and “open succeeds, first write fails” (the 0-byte file case).
3. Optional follow-up, not required to stop the abort: age-clean completed spills, or put the private spill dir under DSH_HOME instead of os.tmpdir(), so a systemd tmpfs user quota cannot kill the host. Raising the OS quota is not a product fix.

Non-goals

• Dropping spill files entirely (already rejected: full-output recovery is a real bash behavior).
• Treating this as a model-picker or session-restore bug. Those were the user-visible symptoms of the host restart.

[argszero]

源码侧全部核实（本地 0.1.2-alpha.1 = cd5ef81，与你引用的 staging 版本同源）。你的故障链每一步都能在源码里对上，且这个 bug 属于一个已知家族——你命中的正是家族此前追踪过的同一函数同一机制，EDQUOT 是第二个触发类。

核实结果（逐条，packages/subprocess/subprocess-local/src/spawn.ts）：

• spillAll（:156-173）：openSync（:169）+ writeSync（:170 追写 prior、:172 写当前 chunk）全部无 try/catch，与你引用的堆栈一致;
• discardSpill（:176-197）确实 catch 了 closeSync（:184）与 unlinkSync（:192）；seal（:227-237）也 catch closeSync——你"close/unlink 已接住、open/write 没有"的观察精确成立;
• 调用链：stream.on('data', (chunk) => { collector.push(chunk) })（:366）→ push（:131）溢出时同步调 spillAll（:134）——data 监听器里抛出的异常无任何兜底，直接 uncaughtException 杀死整个进程，与你的 journal 堆栈（lib/index.js:862）一一对应;
• spill 目录确在 OS tmpdir 下：privateSpillDir() = mkdtempSync(join(tmpdir(), 'dsh-subprocess-'))（:90，0700 每进程私有）; README:130 也明说 "Completed spill files are not deleted… accumulate under the OS tmpdir until something external cleans them"——这个累积机制正是你踩进配额的前提;
• 测试基建与你描述一致：tests/spawn.spec.ts:58-83 已有 vi.mock('node:fs') + failNextClose/failNextUnlink（走 vi.hoisted 开关），扩展 failNextOpen/failNextWrite 完全符合现有模式，两个用例（open 失败 / open 成功首写失败=你的 0 字节文件现场）都写得出。

家族归并（关键增量）：#3190（Windows 11, rc.7）报告的正是同一个函数同一个机制——外部 %TEMP% 清理删掉运行中的 spill 目录 → spillAll ENOENT → 同样 uncaught → 同样崩掉整个 dsh web。你的 EDQUOT 是同一无保护段的第二个故障注入点（外部删除 / 系统配额，殊途同归）。这意味着正确修复面是"spillAll 的同步 fs 失败必须被容纳"，而非单点 EDQUOT：

• 你的 fix #1（catch → discardSpill()，禁用 spill + 保留内存 tail）一举覆盖两个触发类（ENOENT + EDQUOT/ENOSPC），是跨家族的单点修复，强烈支持;
• 家族此前（comment-18070748）对 [Bug] dsh web crashes with ENOENT when external temp cleanup removes the subprocess spill dir #3190 的建议是 recreate-on-miss；你的 degrade 方案更简单且同样安全（进程不死 + 有界 tail 保留）。也可以两者叠加：catch → 尝试重建目录一次 → 仍失败再 discardSpill——全量输出恢复与进程存活兼得;
• fix #3（age-clean / 挪到 DSH_HOME）建议独立跟进：注意 [Bug] dsh web crashes with ENOENT when external temp cleanup removes the subprocess spill dir #3190 的教训是外部清理反而有害（删了运行中目录），若自管目录则必须配套内部 age-clean 策略，属设计讨论、非本次 abort 修复必需;
• 补充一点：df 与配额不一致是 quota 语义，代码里拿不到可靠信号，所以"catch + degrade"确实是唯一正确层，无需尝试探测配额。

一句话：修复 = 把 spillAll 的 openSync/writeSync 包进 try/catch，失败即 discardSpill() 降级，进程永不因 spill 的同步 fs 错误而死；这一个改动同时闭合 #3190（ENOENT）与 #5175（EDQUOT）两个家族成员。PR 通道重开后这是现成的高价值候选。