V8 heap OOM after ~8h: opened Conversations are fully hydrated and never evicted (no live-session retention policy)

[zhangys11]

V8 heap OOM after ~8h: opened Conversations are fully hydrated and never evicted (no live-session retention policy)

Summary

@deepseek-ai/dsh 0.1.2-rc.1 aborts with a V8 heap OOM (FATAL ERROR: Ineffective mark-compacts near heap limit) after
~8 h 16 m of continuous use on a 32 GiB machine with no heap override. Opening a conversation in the web UI cold-reads
that session's entire stored event log, promotes it into a fully hydrated live Session plus a live, resident Agent,
and nothing in the harness ever evicts, caps, or idle-times-out those live sessions — heap usage grows monotonically
with the number of conversations opened until the process dies. Root cause: missing eviction/retention policy for live
Sessions/Agents. Data is not at risk — every session is persisted append-only to disk — this is purely an in-memory
lifecycle gap.

Environment

• OS: macOS (darwin); Node.js v24.16.0 (launcher-captured VERSION file: node v24.16.0, pnpm 11.5.0)
• Launcher & core: @deepseek-ai/dsh / @deepseek-ai/dsh-session 0.1.2-rc.1, installed via
  npm i -g @deepseek-ai/dsh
• Web profile bundles: @deepseek-ai/dsh-base, @deepseek-ai/dsh-web-app, @anionex/dsh-vision-toolkit@0.1.40,
  @linxin666/dsh-web-all@0.3.14, dsh-better-sidebar@0.18.0
• No NODE_OPTIONS / --max-old-space-size set (default V8 heap ceiling ~3.6 GB)
• Machine: 32 GiB RAM

Crash evidence

• Fatal abort:
  FATAL ERROR: Ineffective mark-compacts near heap limit — Allocation failed - JavaScript heap out of memory — V8
  abort via node::OOMErrorHandler, delivered as SIGABRT (crash report shows the abort).

• GC preamble from the aborting isolate (verbatim):

  [12336:0x89f40c000] 29779168 ms: Mark-Compact 3479.5 (3603.4) -> 3465.4 (3603.9) MB, ... (average mu = 0.321)
  

  Uptime 29,779,168 ms ≈ 8 h 16 m. The final mark-compact shrank the live set only from 3479.5 MB to 3465.4 MB
  (~3.47 GB) — reclaiming ~14 MB — so the objects were genuinely retained and reachable, not garbage or fragmentation.

• Crash report on disk: ~/Library/Logs/DiagnosticReports/node-2026-09-05-072536.ips (PID 12336).

• The heap total reached 3603.9 MB — the ~3.6 GB default V8 ceiling — at the time of the abort.

Session scale involved

The store holds a small number of very large single sessions. The heaviest file decompresses to ~127 MB (126.7 MB) of
JSON in 151,184 JSONL rows, yet its event seq counter is ~3,432,683 — rows are packed chunk runs that expand on
decode; when last observed it was at turn 421. Per-session sizes of the 6 largest (paths omitted; only the workspace
context is named):

Raw (MB)	Compressed (MB)	Session context
126.7	48.1	e-commerce workspace session
61.9	25.7	specchain session
60.2	21.9	e-commerce-papers topic_review session
56.7	24.2	piecewise exponential estimator session
55.1	22.3	e-commerce-papers topic_fl session
54.1	19.6	graph session

35 files exceed 2 MB compressed, and the whole store decompresses to ~1.13 GB raw. Opening even a handful of these
resident at once is enough to exhaust a default-sized V8 heap.

Root-cause analysis (code references are from the installed 0.1.2-rc.1 build)

All paths relative to @deepseek-ai/dsh/node_modules/@deepseek-ai/:

• Unbounded live-session store. dsh-session/lib/index.js: SessionStore holds live sessions in an unbounded Map
  (store = new Map(), line ~1544); create() is prepare + enter + announce folded into a ctx.effect (~1579–1586);
  the only removal path is the detach disposer returned by enter() and its paired detachEntered (~1652–1692);
  list() returns everything (~1786–1788). No eviction/LRU/idle-timeout/capacity check exists anywhere in the package.
• Append-only, never-trimmed in-memory log with a second full copy cached. Session starts with log = [] (~1229);
  append() deep-freezes each event and pushes it (~1403–1443); snapshotEvents() caches a second frozen copy of
  the entire log on the first full-range call (eventsSnapshot ??= Object.freeze([...this.log]), ~1342–1348), and that
  cache is invalidated and rebuilt on every append (~1434). Resident cost is therefore roughly one large live object
  graph per opened session plus a duplicate full-length reference array — and neither is ever released while the
  process runs.
• Resume-on-open chain (web client follow with {kind: "session", maxMessages: 50}):
  dsh-api-session-controller/lib/index.js follow (~2815) → history.follow (~1239) reads the observation and, when
  the session was cold/prepared, promotes it (~1296–1304) → promote() fires agents.resolveObservedAgent (~2635–2660)
  → resolve short-circuits if the agent is already live, otherwise resumes (~213–236) →
  dsh-agent-loop/lib/index.js resume (~1349–1370) → persistence prepare seeds the Session with the entire
  stored log (dsh-session-persistence/lib/index.js ~951–965 and ~1161–1178 — whole storedEvents plus repair closers)
  → agent-loop publish enters the live Session and live Agent into their registries (dsh-agent-loop/lib/index.js
  ~1249–1255). So merely opening (even previewing) a conversation fully hydrates it.
• Live agents idle indefinitely between turns. After a turn, kick() returns the agent to {kind: "idle"} and it
  waits for the next prompt (~449–496 / ~605–609); nothing bounds how many such live agents/sessions accumulate (the
  agent AgentRegistry is likewise a bare store = new Map(), dsh-agent/lib/index.js ~416).
• Disposal fires only from owner-context teardown. Disposal runs only from the agent-lifecycle teardown armed on the
  owner context (dsh-agent-loop/lib/index.js dispose ~1201–1221, armed ~1225–1229). Web-driven resumes run under
  the app-root context, so dispose effectively fires only at process teardown
  [inference — the web-resume owner context is the app root; not directly verified in the controller code]. Switching
  conversations, archiving, and idling never dispose: dsh-workspace/lib/index.js archiveSession (~422–432) only
  appends to archivedSessionIds; there is no host "active session" concept and no dispose-on-switch.
• Listing/search are cheap and disk-backed; only OPEN hydrates. Session list reads headers from JSONL plus cached
  projection snapshots with no log read (dsh-api-session-controller/lib/index.js ~1728–1752;
  dsh-session-projection-cache cachedSnapshot ~178–195); full-text search is SQLite FTS. The cold read that decodes
  the whole JSONL happens only on open (dsh-session-persistence-jsonl/lib/index.js readPrefix ~977–1078). Memory
  therefore grows with each conversation you open, not with listing or searching.
• No configuration knob exists to bound resident sessions or idle time. Checked schemas: dsh-agent-loop Config
  only carries maxParallelToolCalls (~1049–1061); dsh-api-session-controller Config only carries
  cold-probe/native-open settings (~2581–2584); dsh-session-projection-cache and dsh-workspace have no Config at
  all. The only LRU in the stack is preparedSessionCacheSize = 5, which bounds detached/prepared (not-yet-published)
  cold sessions (dsh-session-persistence/lib/index.js ~488) — published/live sessions are never evicted.
• Eviction is safe by construction. Re-opening a cold session is already a supported, normal path (cold → prepared →
  promote → resume), which means evicting idle live sessions is fully recoverable from disk.

Reproduction

1. dsh web with default settings (no --max-old-space-size / NODE_OPTIONS override) on a 32 GiB macOS machine.
2. Through the web UI, open several historical conversations with very large transcripts (e.g., a few sessions whose raw
   JSONL is 50–120 MB — millions of decoded events each). Each file is a concatenated multi-frame zstd JSONL, so no
   file-content
   details are needed — only the decompressed size matters.
3. Switch between them and wait minutes/hours. With the Node inspector (Runtime.getHeapUsage / HeapProfiler) observe
   that heapUsed grows monotonically with each session opened and is not reclaimed after switching away,
   archiving, or leaving the conversation idle.
4. Continue opening/using large sessions (or running long agent tasks that resume them). After enough resident sessions,
   the process aborts with FATAL ERROR: Ineffective mark-compacts near heap limit.
5. Expected: idle-eviction, dispose-on-switch, or a resident cap keeps heap bounded. Actual: unbounded monotonic growth
   until OOM.

Suggested fixes (for maintainers)

• Idle eviction: dispose the idle agent-loop and detach the live Session after an inactivity threshold,
  excluding the currently-followed session. Safe because re-opening cold-hydrates from disk (already a supported path).
  Suggest config knobs such as idleTimeoutMs and maxResidentSessions.
• Optionally, LRU/cap concurrent live sessions rather than using a pure time threshold.
• Consider not promoting a live Session + Agent for a mere follow/preview of a conversation — page reads already
  work cold (list, projection snapshots, and message-page reads all avoid the log). Hydrate only when the user
  actually sends a message; this alone would bound memory to actively-used sessions.
• Consider memory-bounding Session.log (windowing / spill) and not caching a second full frozen copy in
  eventsSnapshot for multi-million-event sessions.

Additional context

This was observed while investigating an unrelated plugin regression, not during a stress test. No data was lost:
sessions are persisted to disk as append-only zstd JSONL, so the crash only discarded in-memory state — the on-disk
files are intact.

[argszero]

@zhangys11 你的根因判断我在 0.1.2-rc.1 源码上逐项核对了——成立, 而且是典型的"有 drop 入口但无人自动调用"缺口。补充点源级证据并给一个可落地方向。

确认: live Session 只增不减, 无自动 eviction

会话的 live 实例由 packages/api/session-controller/src/client/sessions/manager.ts 持有, 懒建后永驻 this.sessions Map:

• get()（:288）懒建 createSession() 并 this.sessions.set(sessionId, session), 从不因 idle/超时删除。
• 唯一的外部 eviction 入口是 drop()（:245-249, this.sessions.delete + startSessionDisposal）。
• 而 drop() 的唯一调用方是 service.ts:692 的 dropScope()——它由 sweepDeferred()（service.ts:697）驱动, 而 sweepDeferred 只在stage 移动时被调用（用户导航/scope 切换）。也就是说:只要用户不主动切走那个选中会话, 该 live Session 就一直驻留, 没有任何 idle-timeout 或基于内存的淘汰。

所以你观察到的"打开越多会话堆越高"完全吻合——sessions Map 无界增长, 每个成员都带着完整的 Session + 常驻 Agent。数据本身安全（append-only 落盘, 重开复用 open() 回填历史, get()/drop() doc 里明说了"durable truth 在 host session log, 重开懒建"），纯粹是内存生命周期缺口。

为什么 3.47GB 之后才死

V8 默认 heap ceiling 约 3.6GB（我机器无 override 时也是这个数）。最后的 mark-compact 从 3479.5MB 只回收 14MB 证明对象是真驻留 GC 根引用强持有, 不是漏 GC——所以是 OOM 而不是 leak, 你的"monotonic 增长"判断正确。

建议的落地方向（供参考, 非 PR）

drop 的懒建/可重建语义已经就绪（重开 open() 回填）, 所以加一个自动淘汰不破坏正确性。可选三层:

1. 最省事: 在 manager 加一个 idle-timeout（比如 N 分钟无事件/无 agent run 即 drop(sessionId), 类似 session-title 的 lifetime 模式）。这直接封住"打开很多会话不开走"的路径。
2. 大小上限: 给 sessions Map 加容量上限（如 LRU/score-based, 按最近活动 evict 最久未活跃的）。
3. 统一由 stage 生命周期驱动: 现在的 sweepDeferred 只在"切走"时清, 但"停在一个会话很久"这个场景被漏了。可以在 dropScope 之外补一个对当前 staged 会话的长时间无活动检查。

优先做 1（idle-timeout）——它最小且直接命中你的复现路径; 2 是锦上添花。要注意的是 live Agent 也要跟着 session 一起 dispose（dropScope 里 record.fiber.dispose(), 这个路径已把 agent-loop 的 teardown 兜住了）。

如果你愿意, 我可以基于这个路径出一个具体的 idle-eviction 参考 diff（含 manager idle-check + 回归测试: 打开会话后无活动 N 分钟触发 drop、重开会话回填正常）。

[mirsalimorteza4-hub]

Git switched changes fixedkeep feedback DeepSeek all COANTOMI keeping100%swift Code all work set to chanies New maniging 💪📢🚀💯