在没有 systemd 的主机上， fallbackOwner.waitForExit 会每 15ms 全表扫描一次 /proc

[ShineBreaker]

下面是 DeepSeek V4.1 Flash 写的，大概率会很难看懂

环境

• @deepseek-ai/dsh 0.2.2-rc.1，Node 26.12.1
• Linux / Guix System（没有 systemd），22 核，约 740 个进程
• Web profile，5 个活跃会话；测量期间没有任何 agent 回合在跑

症状

不操作 UI、也没有 agent 活动时，dsh web 的 host 进程长期占 44~52% CPU。Web UI 跟着发卡——每次输入都要排在这部分负载后面。重启 host 能短暂恢复，但只要还有子进程活着，压力很快回来。

测量

strace -f -p <host> -e trace=read 跑 5 秒：76,504 次 read，其中 主线程在 3 秒内对同一个 fd 读了 50,593 次，内容全是 /proc/<pid>/stat（连 kworker/* 这类内核线程都在读）。

通过 inspector 采的 8 秒 V8 CPU profile，调用链是：

readFileSync <- readLinuxStat <- linuxProcessGroupHasLiveMembers <- alive()
  <- waitForExit <- (root)

self time：readFileUtf8 21.4%、parseProcStat 8.6%、readdir 4.0% —— 光这一条链就占了约 34% 的采样。

根因

packages/subprocess/subprocess-local/lib/runner-launch-*.js：

function sleepTick() {
  return setTimeout$1(15);
}

waitForExit: async () => {
  ...
  observation ??= (async () => {
    while (alive()) await sleepTick();
    stopped = true;
  })();
  await observation;
}

const alive = () => {
  ...
  process.kill(-pid, 0);
  if (directSettled && platform === "linux" && linuxGroupHasLiveMembers(pid) === false) return false;
  return true;
}

function linuxProcessGroupHasLiveMembers(processGroupId, internals = DEFAULT_INTERNALS) {
  entries = internals.readDir("/proc");
  for (const entry of entries) {
    if (!/^\d+$/.test(entry)) continue;
    const stat = readLinuxStat(internals, Number(entry));
    ...
  }
}

在拿不到 systemd scope 的主机上会选 fallbackOwner。直接子进程 settle 之后，每次 alive() 只要 kill(-pid, 0) 成功就会做一次全表扫描，而 waitForExit 每 15ms 调它一次。于是 N 个进程 → 每 15ms N 次文件读；736 个进程的机器上实测约 9,000 次 open/秒。

建议的修法

1. 给轮询加退避：15ms 起步，进程组一直活着就退到 250ms~1s。退出检测只要亚秒级就够，不需要 66Hz。
2. 采样昂贵的那一步：process.kill(-pid, 0) 已经能回答「进程组还在不在」；把 linuxProcessGroupHasLiveMembers 限制成每个 owner 每几百毫秒最多跑一次。
3. 更彻底：不再全表扫描。后代树用 /proc/<pid>/task/<tid>/children 递归就够，根进程存活用 pidfd，全表只留给「只剩僵尸」这个边角情况。

本地 workaround

我把 sleepTick 提到 100ms，并给进程组扫描加了 250ms 节流。同一台机器实测：host 空闲 CPU 从 46% 降到 2~3.5%，UI 延迟恢复正常。改动由一个幂等脚本在每次安装后自动重打。

补丁 diff 可以随时提供——不过按 CONTRIBUTING 的说法目前不接受 PR，所以先发在这里。

Related: #12353 是同一模块在 systemd 路径上的对应问题（transient scope 累积把 user manager 拖满），我这边是无 systemd 时的 fallbackOwner 路径。

[PerryLink]

这条是真的：waitForExit 每 15 毫秒醒一次，醒来就扫一遍 /proc 全表。

链路对得上

• packages/subprocess/subprocess-local/src/spawn.ts:88-90：sleepTick() 就是 sleepMs(15)。
• :243-252：waitForExit 里 while (alive()) await sleepTick()。
• process-inspector.ts:207-226：linuxProcessGroupHasLiveMembers() 先 readDir('/proc')，再对每个数字条目调 readLinuxStat 读 /proc/<pid>/stat，只看 pgrp 匹配的项。

所以你 strace 出来的"3 秒里主线程对同一个 fd 读了 5 万次"完全对得上：740 个进程 × 每秒约 66 次 = 每秒约 4.9 万次 stat 读。

为什么要扫全表

因为 alive() 第一步的 process.kill(-pid, 0) 在 Linux 上只要组里还有僵尸就会成功，所以代码必须再看一眼状态：process-inspector.ts:223 只把非 Z/X/x 的成员当活的。这一步就退化成全表扫描了。

这条路径什么时候跑

不是"有 systemd/没 systemd"本身触发的，而是 alive() 里 directSettled 为真、组里还有成员的时候才走到这个扫描。没有 systemd 帮你及时回收子进程时，这个状态会持续更久——所以你的机器上看起来像"常态"。

修法形状

1. 让 tick 退避：15ms 对一个"等退出"的循环来说是没必要的精度，至少改成逐步拉长到 100–250ms；
2. 别扫全表：只检查这个组自己 spawn 过的 pid（进程表里记着），或者用 pidfd；
3. 真要看组：/proc/<pid>/task/*/children 沿着树走，比遍历 740 项便宜得多。

只读了 master fcc0ad46 的源码；你的 strace/profile 我没法复算。

[sashatroshin]

Adding a field report from a production container (no systemd, Docker), because the user-visible impact is worse than "UI feels slow".

Environment: @deepseek-ai/dsh 0.2.1-rc.4, Node 24.21.0, Debian container, 4 cores, ~2100 processes, ~1900 session directories, web profile.

Impact on the Web GUI. With the fallback waiter spinning, POST /api/session/list never returns. The sidebar renders the workspace group with zero session rows and the main pane shows "Choose a workspace to start" — it looks like every chat was deleted, with no console error and no failed request in the network tab. Data was intact the whole time. This lasted ~3 days on an always-on host.

Measurements (same host, before → after killing the leaked process group):

Metric	waiter spinning	waiter gone
host CPU	100% of one core (301–303 ticks / 3 s)	1–6% idle
read syscalls	~252k per 3 s (~84k/s ≈ 40 full /proc scans/s)	~0
bytes read	~22.5 MB per 3 s	~0
rchar since boot (3.3 d)	705 GB	—
POST /api/session/list	never responds	1.8 s, 4.6 MB, ~1480 sessions

V8 CPU profile matched yours exactly: readFileUtf8 74%, readdir 11%, parseProcStat 8%. Breakpoint on linuxProcessGroupHasLiveMembers returned the process-group id that was being polled, which is how I identified the culprit: a python3 -m http.server whose parent shell had exited 38 h earlier, plus three orphaned node /tmp/*.mjs scripts.

Root cause of the leak (no bug in alive() itself). Any background command that daemonizes — a preview HTTP server, a dev script — leaves a live descendant after its shell exits. The group therefore never empties, so one waiter per leaked group spins forever, one core each. Killing the orphaned descendants is the only external remedy; there is no API to cancel that wait.

What made it much worse in practice. The cold session catalog is expensive (1900 session dirs, and every list also recomputes historicalCorpusRevision because 14 legacy v3 logs exist), so it needs a few seconds of an unstarved event loop. One spinning waiter is enough to starve it indefinitely, which is why the GUI symptom is "all chats gone" rather than "a bit slow".

+1 for the throttling/backoff fix in the waiter, and for not scanning the whole process table per tick. Happy to test a patch on this host.

[2SinceJuly]

@PerryLink's trace is correct, and there's a specific reason the cost is unbounded per poll that's worth adding — plus a fix that costs nothing in correctness.

The fast path can never engage for this workload. linuxProcessGroupLiveMember has a knownMember short-circuit at the top (process-inspector.ts:207-226): if the remembered PID still has the right pgrp and isn't Z/X/x, it returns immediately without touching /proc. fallbackOwner.alive() passes liveMember on every poll (spawn.ts:226-232) and updates it whenever a live member is found.

But look at the guard that reaches it: if (directSettled && platform === 'linux'). directSettled flips true only when the direct child has exited. So the enumeration exists to answer "did the descendants outlive the direct child" — and while it's answering no, every iteration pays a full /proc walk. The knownMember cache only helps in the narrow window where the direct child is gone but a descendant is still running, which is not the long-lived case. In practice, on a host with ~740–2100 processes and a group that keeps a member alive, you get a full table scan every 15 ms forever. That matches the field report of ~50k stat reads in 3 s.

Three fixes, cheapest first:

1. Reuse the last matched pid, not just the last live one. Today liveMember is only assigned when a non-quiescent member is found. Remember the pid of any entry with a matching pgrp regardless of state, and on the next poll read only that one /proc/<pid>/stat. Zombie members have a stable pid too, so a group of one zombie plus one live child resolves in one read instead of N. This is a two-line change to linuxProcessGroupLiveMember and doesn't weaken anything: the full scan still runs whenever the remembered pid disappears or changes pgrp.

2. Back the cadence off. sleepTick() is a hardcoded sleepMs(15) (spawn.ts:88-90). The comment explains it's ref'd deliberately so teardown can't orphan survivors — that's about keeping the timer alive, not about the interval. A 15 → 100 ms start with a geometric ramp (e.g. ×1.5 capped at 1 s) keeps worst-case teardown latency under a couple of seconds while cutting idle polling by ~15×. A teardown that takes 900 ms instead of 60 ms is a much smaller problem than 50% CPU on an idle host.

3. Prefer pidfd where available. process.kill(-pid, 0) is the group-level primitive, but for the direct child Node exposes an exit event, and for descendants pidfd_open + poll gives completion notification with zero scanning. That's a larger change and Linux-only, but it's the only version that's O(1) rather than O(procs) in the steady state.

I'd ship (1) and (2) together — (1) alone still leaves a 15 ms poll doing one stat read, and (2) alone still does a full scan per poll, just less often.

On the container report from @sashatroshin: the reason POST /api/session/list never returns is likely the same loop holding the event loop rather than a separate bug — readFileSync in a 15 ms setTimeout chain starves the poll phase that would resolve the request's I/O. Worth confirming with --cpu-prof while the session list is hanging; if the profile shows the same readFileSync <- readLinuxStat chain, it's one bug with two symptoms and one PR.

[2SinceJuly]

One correction to the fix, since it changes where the poll loop should live.

waitForExit polling every 15 ms isn't the real cost driver on a busy host — readDir('/proc') plus one stat per PID is O(processes) per poll, so at ~2100 processes and ~67 polls/s that's ~140k syscalls/s just to answer "is this PID alive". The interval is only the multiplier; the fan-out is the problem.

Two fixes, and the second is the one I'd propose:

1. Adaptive interval — back off (15 ms → 50 ms → 200 ms) with a hard ceiling, reset on state change. Cheap, bounded, no correctness change, but it's still O(processes) per tick.
2. pidfd_open where available — Linux 5.3+ gives a file descriptor you can poll() on, which turns "is it alive" from a /proc scan into a single event. Fall back to the /proc path when pidfd_open fails. On the Docker/no-systemd target you describe, that removes the scan entirely rather than making it less frequent.

If only one lands, take (2) — (1) makes the symptom quieter without fixing the scaling, and on a 2100-process host the interval would have to go to seconds to matter, which breaks the responsiveness the 15 ms was chosen for.