Replies: 1 comment
|
Follow-up with more precise trigger characterization (heap limit 2560 MB, WebSocket heartbeat raised to 20 s via the gateway config override):
So the retention lives in the per-open consumers of the decoded log (downstream of |
Uh oh!
There was an error while loading. Please reload this page.
Environment
0.1.5-rc.1and0.1.5-rc.2(via npm;dsh webprofile, shipped bundles only)What happens
dsh webboots fine and serves the UI, then the server process dies with a V8 heap OOM after roughly 30–90 seconds of session activity:The heap climbs to the V8 default limit (~1.9 GB) and mark-compact cannot reclaim it (
1936.9 (1947.6) -> 1935.1 (1942.9) MB). WithNODE_OPTIONS=--max-old-space-size=1024it OOMs in ~28 s; with the default limit in ~83 s. Raising the limit only postpones the crash proportionally.The trigger is opening/switching sessions in the web UI — or the frontend auto-reconnect re-opening the last session. A stale client that reconnects every ~3 s (e.g. an auto-retrying port-forward with the tab still open) will crash a freshly booted, otherwise idle instance in ~80 s with no human interaction.
Control experiment: an instance whose webserver failed to bind (EADDRINUSE) sits flat at ~383 MB RSS for many minutes — the leak requires a successfully served UI + session loads.
Evidence (V8 sampling heap profile)
Captured with
HeapProfiler.startSampling(512 KB interval) during the growth window, ~170 s:97.5% of all sampled allocation lands in
consumeEventLine(theSessionLogScannerpath:JSON.parse(line)+restore.decodeRow(decoded)per JSONL row).The session in question is small (~3.8 MB of JSONL), so a single decode cannot fill the heap — the observed ~35 MB/s sustained allocation rate with net retention implies the log is re-decoded on every session open / reconnect and the decoded copies are retained long enough to ratchet the heap up until OOM.
Reproduction
dsh web --no-openin a workspace that has a session with historyExpected
Repeated opens of the same session revision should reuse the memoized decode (or release prior decoded copies). The heap should stay flat across open/close cycles.
Notes
coldLogMemois bounded (COLD_LOG_MEMO_MAX_ENTRIES = 2), so the retention appears to live in the per-open consumers downstream ofdecodeStoredLog/scanLog, not in the memo itself.dsh-session-persistence-jsonl/lib/index.jsis byte-identical between rc.1 and rc.2, so upgrading does not fix it.All reactions