Host OOM crash loop at startup: cold session projection eagerly full-reads every session log

[lxfebd]

Summary

The desktop host (Electron 44.7.0 / dsh-desktop runtime 0.2.1-rc.2) repeatedly crashes with FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory during startup session projection, then the parent respawn loop surfaces as the GUI error "应用无法启动或已意外停止". The crash is triggered purely by accumulated session-log size, not by any single conversation.

Root cause

At cold start, materializePersistenceSnapshots in dsh/node_modules/@deepseek-ai/dsh-session-query-sqlite/lib/index.js iterates persisted.values() and for every session calls readColdSessionLog(...), which does handle.read(0, void 0) (full read of the entire session log) in dsh/node_modules/@deepseek-ai/dsh-session-query/lib/index.js.

• Each session is stored as session.v4.jsonl.zstd (multi-frame zstd JSONL): the first frame is the session header/card, later frames are event bodies.
• The heap is pinned at the V8 old-gen cap (~3.7–4.3 GB). When the sum of all session logs exceeds that during the eager full-read, allocation fails → exit 134 → respawn loop.

Suggested fix (matches the intended UX)

Make startup projection read only the first frame (header/card) per session, and lazily load full event bodies when a session is actually opened (unload on close). The building blocks already exist in-repo: readFirstZstdLine / readFirstLine in dsh/node_modules/@deepseek-ai/dsh-session-persistence-jsonl/lib/index.js already read only the header frame.

This keeps conversation cards visible at startup while full content stays unloaded until viewed — exactly the "card stays, content loads on demand" behavior, and it permanently removes the OOM loop as session history grows.

Repro environment

• DSH desktop host, current production bundle (runtime @deepseek-ai/dsh-desktop 0.2.1-rc.2)
• Windows, Electron 44.7.0, bundled node 24.21.0
• Crash signature: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory; respawn at lib/main.js:3724:64

[argszero]

Thanks — the read amplification is real and your reasoning about it is close. I traced it against the 0.2.1-rc.2 tree (687f282e86, the release you are running) and found two things worth correcting: the entry point that reaches the read, and the fix shape. Both matter, because the fix you propose would leave the index empty.

Where the whole-log read actually lives

materializePersistenceSnapshots is only the map builder; it never reads a log. The read is one loop above it, in the SQLite backend's reconcile:

• packages/session-query/session-query-sqlite/src/index.ts:513 — for (const entry of persisted.values())
• :522 — const loaded = await readColdSessionLog(persistence, entry.header.id, signal)
• packages/session-query/session-query/src/cold-read.ts:40 — handle.read(0, undefined, options), the whole contiguous log

Two skip conditions guard that loop (:515–:521): the session must not be live, and its stored revision must differ from the indexed row. So the cost is bounded by sessions whose log changed or was never indexed, not by the corpus — which is exactly why a cold index is the expensive case and a warm one is cheap.

One correction: the startup list cannot reach this loop

ApiSessionList.list() (packages/api/session-controller/src/list.ts:132) goes to sessionQuery.listSessions (:134), which is header-only: the persistence listing reads just the first frame per log (packages/session/session-persistence-jsonl/src/index.ts:1179 → readFirstZstdLine at :1497, bounded chunks, one frame decompressed), and cold rows take their projections from the durable cache by header alone — "missing cells and cache rows are never materialized here" (list.ts:327). There is no path from the card list to readColdSessionLog.

The reconcile runs only from searchSessions (index.ts:268) and searchEvents (:296), each via _ensureReady then _reconcile. So a crash at startup means something in your install issued a search. With the shipped base and web-app rows that is impossible — both mount path: ':memory:' with openAt: never (packages/bundle/base/cordis.patch.yml:157, packages/bundle/web-app/cordis.patch.yml:27), and openAt: 'never' throws SESSION_QUERY_SEARCH_DISABLED before any SQLite work. Enabling content search means adding @deepseek-ai/dsh-experimental-session-search, whose row is path: ':memory:' + openAt: first-search (packages/experimental/session-search/cordis.patch.yml).

That :memory: is the part your report is missing, and it is the likely root cause of the recurrence: an in-memory index is rebuilt from nothing on every process start, so _observeStable sees an empty persisted_sessions table each launch and the "unindexed" branch is taken for every session. Cold read is not the startup case — with this backend it is the steady state.

Why it exhausts the heap rather than just being slow

Per cold session, observeSession (index.ts:876) builds four representations and keeps the third:

1. loaded.events — the full log, from readColdSessionLog (plus interruptedTurnClosers appended)
2. detachedEvents = events.map(event => structuredClone(event)) (:882) — a second full copy, event by event
3. documents: buildSessionEventSearchDocuments(...) (:886) — the extracted text of every message event, retained
4. createHash('sha256').update(JSON.stringify({ header, inheritedEventCount, events: detachedEvents })) (:887–:890) — one contiguous string holding the entire serialized log, materialized in a single allocation

The persisted map holds entry.loaded for every cold session until the reconcile transaction commits, so the retained document set is O(sum of all logs), while the reads themselves are sequential — one session at a time, persistedReadConcurrency does not apply to this loop. A single allocation failing inside that loop is precisely CALL_AND_RETRY_LAST Allocation failed. STABLE_OBSERVATION_ATTEMPTS = 2 also lets the whole pass repeat if the corpus changed underneath it.

The fingerprint in (4) is dead work on this path: change detection for persisted entries is revision, and _replacePersistedSession never reads entry.fingerprint (only the live rows use it, :621). So every cold-read session pays a full-log JSON.stringify plus a SHA-256 over it for a value nobody consumes.

Why reading only the first frame does not work as the fix

The cold read exists to build FTS documents, and those come from events, not from the header. A header-only observation yields an empty documents array, so every session would be indexed with no searchable text — search would silently return nothing for cold history. readFirstZstdLine is the right primitive for the card path, but the card path already uses it and already has no cold read. Applying it here trades an OOM for an empty index.

What I would change instead

Ranked by cost:

1. Your immediate workaround: keep content search on but give it a durable path (a file under the profile, openAt: first-search). The revision check then skips every unchanged session, and the second launch reads nothing. On an existing install this converts the crash loop into a one-time indexing cost — but it does not remove the peak on the first run, so pair it with (3).
2. Cheap core fix, no behaviour change: bind entry.revision as the persisted fingerprint instead of hashing the log, and drop the redundant structuredClone on this path (the cold read already returns caller-owned events). That removes roughly two full-log materializations per session and the single-allocation JSON.stringify that most plausibly triggers CALL_AND_RETRY_LAST.
3. The actual fix: bound one reconcile pass by bytes or session count instead of "every unindexed session", carry a persisted high-water mark so the rest is indexed on later passes, and keep the FTS inserts chunked across transactions. That makes peak memory O(largest single session) rather than O(corpus), which is the only shape that survives history growth. Search would return partial results with a pending signal until the corpus is fully indexed — an honest degradation, unlike an unbounded first pass.

What I would like to know to narrow the trigger

• Which bundle or config override enabled content search in your install — @deepseek-ai/dsh-experimental-session-search, or a --patch/profile override of openAt?
• Does the crash happen before you type anything into the sidebar search, or on the first search after launch? The client only issues a search once the box is non-empty (packages/client/ui-workspace/src/client/rows/WorkspaceBrowser.tsx:1153), so if it is literally pre-input, something else in the composition is opening search and that component is worth naming.
• Total size of your sessions tree and the largest single log — that separates "sum of many logs" from "one log too large for the constant factor above", which decides whether (2) alone would have saved you.