Cold-opening a long session costs seconds: the whole log is decoded before the tail page is served

[Wangmo-bit]

Summary

Opening a cold long session (one not currently attached) in the Web GUI takes roughly 2–3 seconds here. From reading the shipped packages, the cold read path materializes and validates the entire event log before it can serve the bounded tail page the client asks for. For one representative session, pure decompress + JSON.parse alone is ≈1.5 s.

This is a report with measurements and one suggested direction, not a request for a specific design.

Environment

• DSH 0.2.2-rc.1 (installed packages: dsh, dsh-session-persistence-jsonl, dsh-session-persistence, dsh-session-query, dsh-api-session-controller all 0.2.2-rc.1), Web profile, Linux.
• Session format v4 (session.v4.jsonl.zstd, zstd frames).
• Measurements are read-only shell/zstd/node timings over local session files; no session content was read (sizes, frame counts, and parse timings only). One machine, not a profiler run.

Measurements

Across 659 local session logs (690 MB total), the median log is 0.43 MB; 9 are ≥5 MB and 5 are ≥20 MB — i.e. only a subset is affected, matching the "some long sessions" observation.

Representative log:

metric	value
compressed	68.8 MB
decompressed	191.5 MB
events (JSONL lines)	43,332
zstd frames	22,113
zstd -dc to /dev/null	0.40 s
line-by-line JSON.parse (stream)	1.12 s
decode + parse	≈1.5 s

≈2 events per frame; per-frame overhead dominates the decode time.

Where the cost is (from the shipped code)

• session/follow and session/events open an observation; for a cold session the observation reads the whole artifact: readStableJsonlFile + readZstdPrefix (dsh-session-persistence-jsonl).
• Every event is then validated and frozen: validateStoredEvents (dsh-session-persistence), adoptSessionEvent + deepFreeze (dsh-session), plus seed validation and the surface fold when the session is restored.
• Only after that does paginate (dsh-api-session-controller) slice the requested tail page.
• The client's first page is already bounded (≥50 messages and 2 turn starts, cap 500 messages), so the wire payload and rendering are not the bottleneck; the host-side full materialization is.
• Related but not sufficient: the persisted projection cache (session-projection-cache) accelerates projection folds only; the prepared-observation cache (preparedSessionCacheSize, default 5) makes repeat opens of the same cold session fast, but evictions and restarts fall back to the full read.

Suggested direction (one option)

1. Tail-oriented read. zstd frames are independently locatable. A small per-frame index (first event seq per frame, or the zstd frame index) would let a page read decode only the frames it needs instead of the whole file. For this log that is ≈1.5 s → tens of ms for a tail page.
2. Larger write frames. Batching more events per frame reduces per-frame decode overhead; this composes well with (1), where random tail access is served by the index rather than by tiny frames.
3. Slice validation for read-only pages. Validate/freeze the decoded tail slice for viewing; keep full validation for resume/attach paths.
4. Avoid whole-array structuredClone in query paths that return only a slice.

Reproduction (read-only)

# frame count and decompress time (substitute a large session file)
zstd -l <session>.v4.jsonl.zstd
time zstd -dc <session>.v4.jsonl.zstd > /dev/null

# parse time without printing content
zstd -dc <session>.v4.jsonl.zstd | node -e '
  let n = 0, b = "";
  process.stdin.setEncoding("utf8");
  process.stdin.on("data", c => { b += c; let i; while ((i = b.indexOf("\n")) >= 0) { const l = b.slice(0, i); b = b.slice(i + 1); if (l.trim()) { JSON.parse(l); n++; } } });
  process.stdin.on("end", () => console.log(n, "events"));'

Caveats

• Numbers are from one machine; the split between host decode, validation, projection hydration, and client render is inferred from code paths, not measured inside the app.
• Not a regression report: I have no earlier baseline.
• Session contents and local paths are deliberately excluded.

中文摘要

冷启动打开一个“当前未附着”的超长会话约需 2–3 秒。从已安装包看，宿主在提供有界的尾部一页之前，会先把整份日志解码、逐事件校验并冻结；实测代表样本 68.8MB（解压 191.5MB、43,332 事件、22,113 个 zstd 帧），仅解压 0.40s + 逐行 JSON.parse 1.12s ≈ 1.5 秒。客户端首屏本来就限 ≤500 条消息，瓶颈在宿主侧全量物化。建议方向：写入时记录“每帧首事件 seq”小索引，读取只解码尾部所需帧；放大写入帧；只读页只校验切片；查询路径不要整数组深拷贝。以上为只读测量，未读取会话内容。

[BB-84C]

Adding a second consumer of the same root cause, with measurements: incremental reads from a plugin through the public persistence API. The frame index you propose would fix this case as well.

Use case

A plugin keeps a derived view per session (goal history, todos, child-session summaries) and refreshes it periodically. Long-lived agent teams make logs grow without bound, so the plugin is written to be incremental:

• live sessions are folded from session/event;
• cold sessions are only stat-ed; if the revision is unchanged, nothing is read;
• on a revision change it remembers the last logical offset and calls open(id, 'read') → read(offset), folding only the new suffix.

The work is incremental at the logical level, but the physical cost still scales with total log size.

Why (dsh-session-persistence-jsonl 0.2.1-rc.2)

• The read handle's readCurrent first calls storage.readStoredLog(path, id) and then returns events.slice(offset, offset + length) (lib/index.js L103–110).
• readStoredLog reuses the memo only when the file revision is unchanged. Any append changes the revision, so the next read(offset) goes through readStableJsonlFile and decodes the whole file again, followed by validateStoredEvents and freezing every event (L2826–2876).
• For a cold session, offset therefore only controls how many events come back. It does not reduce how much is read or decoded.
• The synchronous event accessors on Session (eventAt / snapshotEvents …) are @deprecated … new calls are prohibited (dsh-session types L208/218/227), so they are not an alternative.

Measurements

Copy of one real log: 37,292,232 B, 18,280 events, zstd. Writer and reader are separate backend instances.

operation	time
stat, revision unchanged (30×)	mean 1.77 ms, no read handle opened
first cold full read	1013 ms
after appending 1 event, read(offset) returning that 1 event (3×)	807 / 817 / 832 ms
same revision, empty tail read	1.4–1.9 ms (memo hit)

Returning one appended event costs as much as reading the whole log. Logs in long-running sessions grow, so this cost keeps rising.

What would help (for the maintainers to decide)

Any public way to read only what was appended, at a cost proportional to the new data:

1. Make read(offset) physically skip the confirmed prefix. Your per-frame "first event seq" index would do this.
2. Or support cursor-based resumption: read returns an opaque cursor (revision plus physical position), the next call reads only later appends, and the call fails clearly if the prefix was rewritten.
3. Or provide a cross-instance append notification, so subscribers don't need to re-read cold sessions at all.

Our current workaround: at most one full read per cold session per 15 s, triggered only when the revision changes, plus a persisted derived checkpoint so a restart doesn't refold. It works, but the cost grows with the log.

---

中文摘要：补充同一根因的另一种场景。插件通过公开的持久化接口做增量读取：冷会话按 revision 判断是否变化，变化后只调用 read(offset) 读新增部分。但任何追加都会改变 revision、让 memo 失效，read(offset) 仍会整份解码、校验、冻结后再切片。实测 37MB / 18,280 事件，只追加 1 条，读出这 1 条要约 0.81 秒。希望提供成本与新增数据量成正比的公开读取方式，例如本帖提出的分帧索引、基于游标的续读，或跨实例的追加通知。

[WanchengWildMan]

Thanks for the detailed write-up. I measured the same frame-count effect synthetically and want to add one data point plus a scoping nuance.

Synthetic benchmark. On a 1,000-frame JSONL+Zstd log, creating a fresh Zstd decoder per frame made a 1,000-event tail read take ~10 s. Reusing one decoder across a contiguous frame batch and extracting only the final N lines with native Buffer operations took ~9–18 ms. The frame count, not the payload size, dominated.

Nuance. Current master already reuses one decoder across the whole frame batch: readZstdPrefix in packages/session/session-persistence-jsonl/src/index.ts and decodeCurrentGeneration in packages/session/session-persistence-jsonl/src/generation.ts call createZstdFrameDecoder() once and pass all scanZstdFrames ranges to decode(). So the ~10 s figure describes a hypothetical tail path that creates a decoder per frame; it is not a bug in the full-read path.

Missing capability. What upstream does not have is frame-indexed tail selection: use scanZstdFrames ranges (or a persisted first-seq-per-frame index) to skip the prefix, then serve the first page from the persisted projection checkpoint without decoding or materializing the whole log. That makes cold open proportional to the requested tail rather than total log size.

Would maintainers welcome a PR for the frame-indexed tail read plus checkpoint-served first page? Happy to prototype it against the current persistence seam and cover the index/checkpoint invariants with tests.

[WanchengWildMan]

Thanks — the code-point analysis matches current master. One correction and one addition.

Correction on the decoder. Master already reuses a single decoder across the frame batch: readZstdPrefix and decodeCurrentGeneration create one frame decoder and pass every scanZstdFrames range to it. Per-frame decoder creation is therefore not the cost on the shipped path. The cost is that a cold read still reads the stable file, validates and freezes every event, and only then lets paginate slice the requested tail.

What is still missing. Frame-indexed tail selection. scanZstdFrames already produces the ranges needed to skip the prefix, and the first page could additionally be seeded from the persisted projection checkpoint when its revision matches. That makes cold open proportional to the requested tail rather than to total log size.

The same restore runs elsewhere. Full materialization is not unique to opening a Session: the first prompt after a resume and model selection on a cold Session both go through resume and pay the same cost. A tail-oriented loader used by resume fixes those cases too.

Would a PR that adds the frame index plus checkpoint seeding be welcome? I would keep the durable format unchanged, deriving the index on first read and caching it per generation.

---

中文说明

感谢——对代码点的分析与当前 master 一致。这里有一处更正和一点补充。

关于 decoder 的更正。 Master 已经在整个帧批次上复用单个 decoder：readZstdPrefix 与 decodeCurrentGeneration 只创建一个帧 decoder，并把所有 scanZstdFrames 区间传给它。因此「每帧新建 decoder」并不是已发布路径上的开销。开销在于冷读仍然要读取稳定文件、校验并冻结每个事件，之后才由 paginate 切出请求的尾部。

仍然缺失的部分。 按帧索引的尾部选择。scanZstdFrames 已经产出跳过前缀所需的区间，首页还可以在持久化投影检查点的 revision 匹配时由该检查点直接提供。这样冷打开的代价与请求的尾部成正比，而不是与日志总大小成正比。

同样的恢复也发生在其他路径。 全量物化不只出现在打开 Session 时：恢复后的第一条 prompt，以及在冷 Session 上选择模型，都会走恢复流程并付出同样代价。让恢复使用面向尾部的 loader 也能修好这些场景。

是否欢迎一个加入帧索引并用检查点预填首页的 PR？我会保持持久化格式不变，在首次读取时推导索引并按 generation 缓存。