[Performance] Switching sessions re-decodes the whole session log (O(history), 22.8k zstd frames) and stalls the desktop UI for seconds

[vickwv]

Summary

On Desktop 0.2.2-rc.1 (Linux, KDE Wayland), switching sessions with long histories freezes the UI for several seconds, and it gets worse as sessions grow. The client only asks for the last ~500 messages, but the host appears to decode the entire session log — every zstd frame, every event — on the main process event loop.

Environment

• @deepseek-ai/dsh-desktop 0.2.2-rc.1, Linux (Arch, KDE Plasma / Wayland), Intel Arc B580 (xe), 14 cores / 32 GB
• Session directory: ~/.dsh/sessions/<workspace>/<id>/session.v4.jsonl.zstd
• Largest affected session: 31,224,297 B compressed → 94.8 MB plaintext / 42,405 JSONL records / 22,793 zstd frames

Measurements

What	Result
Session artifact	29.8 MB compressed → 94.8 MB / 42,405 lines / 22,793 zstd frames
zstd -dc alone	211 ms
JSON.parse every line (bare Node)	342 ms
App cgroup CPU during a ~40 s window of session switching	24 s CPU → ≈60 % of one core average, 100–150 % peaks
App cgroup CPU while idle (no user action)	still bursts to 100 % of one core for 4–5 s
Memory PSI / swap / GPU resets / crashes	all zero → not a resource-pressure problem; other apps on the same machine stay smooth

Because the work is on the host's single JS thread, typing/scrolling in the window stalls in ~0.5 s slices for the whole load. Other applications are unaffected.

Code path (master as of 2026-11-04, fcc0ad46)

1. Client — packages/api/session-controller/src/client/sessions/session.ts
   • open() pulls a tail page only: PAGE_MESSAGES = 50, HISTORY_PAGE_OPTIONS = { maxMessages: 500, ... }, then loadOlder() pages backwards on demand. The client is already incremental.
2. Host — packages/session/session-persistence-jsonl/src/index.ts
   • open(id, 'read') (line 388) → requireStoredLog() → readStoredLog() (line 801):
     • readStableJsonlFile(path) reads the whole artifact into memory;
     • readZstdPrefix() (line 994) runs scanZstdFrames(buffer) and then decodes all frames, feeding SessionLogScanner, which parses every event;
     • validateStoredEvents() validates every event;
     • only then does the history layer pick the requested tail page.
3. Cooperative yield — ZSTD_DECODE_YIELD_INTERVAL_MS = 500
   • the decode loop only await scheduler.yield()s between frames and every 500 ms. For a cold 94.8 MB artifact that is seconds of CPU handed back to the loop in ~500 ms chunks, which is exactly the input-hitch profile users see.
4. coldLogMemo is keyed per session id and revision, so reopening the same session is cheap — but every newly opened session pays the full cost again.

So a tail-page request costs O(entire history).

Why this is avoidable

The log is already stored as a sequence of independent zstd frames (22,793 of them). The last page only needs the last few frames:

1. Tail-first read — seek to EOF and decode frames backwards until the requested maxMessages / turn window is satisfied; keep the forward scan for full-history operations (export, stats, migration).
2. Persisted frame index — store frame offset → seq range per session so open becomes O(index + page) instead of O(history).
3. Finer yielding — ZSTD_DECODE_YIELD_INTERVAL_MS of 500 ms guarantees up to half a second of unresponsiveness per slice; 8–16 ms (or yield every N frames) would keep the desktop responsive during cold loads.
4. Worker decode — lib/worker.cjs already exists for migrations; cold decode + scan could run there and return only the page.

Reproduction

1. Build a session with ~40k events (e.g. long agent runs with large tool outputs — nested tool results, diffs, logs).
2. Switch between that session and another one repeatedly in the desktop app.
3. Observe multi-second stalls while the host decodes the whole artifact; a desktop CPU profile of the switch shows the time in readStoredLog → readZstdPrefix → SessionLogScanner.

Workaround for users

Move very large old session directories out of the workspace to avoid the worst case (keep the .jsonl.zstd; moving the directory is enough — do not delete history you still need).

---

Filed as a Discussion because Issues are disabled on this repository. Happy to provide the profiling trace or test a patch if useful.