Session log reader hard-fails when a complete frame contains a torn trailing JSONL record, rejecting the entire session history

[ybkin1]

Summary

@deepseek-ai/dsh-session-persistence-jsonl stores each session as a concatenation of zstd frames, where every frame carries a batch of JSONL records (the first frame is a header). When the writer is killed mid-append, the last record of the final complete frame can end up inside the frame without its trailing newline. In that case readZstdPrefix() throws corrupt Zstandard session log: complete frame contains a torn JSONL record, readPrefix() fails, and the entire session history becomes unreadable — even though every complete record before the damaged tail is perfectly valid. Structurally torn final frames already have a tolerance/recovery path; this adjacent damage class does not, and one missing newline byte currently poisons a whole session.

Background & Evidence

• Format: <workspace>/<session-id>/session.jsonl.zstd is a concatenation of zstd frames (magic 28 b5 2f fd); each frame holds a batch of JSONL records; the first frame holds a single header line.
• The zero-tolerance branch: after feeding all structurally complete frames to SessionLogScanner, the reader compares checkpoint() values and hard-throws when committedBytes !== inputBytes, i.e. when the plaintext of the last complete frame ends with an unterminated JSONL record (readZstdPrefix() in dsh-session-persistence-jsonl/lib/index.js, ~L983-1011 in 0.1.1-rc.2).
• Field evidence: on one Windows deployment, non-graceful process terminations during normal operation (force-kill, crash, power loss — there is no shutdown handler that finishes or rolls back the in-flight frame) accumulated damaged session logs. 180 of 267 session.jsonl.zstd files failed to load with exactly this error. A local repair that truncates the damaged tail and re-appends the complete records restored all of them with zero event loss — i.e. the data was recoverable and the hard rejection was over-strict.

Failure Mode or Reproduction

1. Start dsh web and open a session so records are being appended.
2. Terminate the process non-gracefully (e.g. taskkill /F on Windows, OOM kill, power loss) while a frame append is in flight, such that the last record's bytes reach the frame but its trailing newline does not.
3. Relaunch dsh and open the same session.

Observed: the session fails to load with corrupt Zstandard session log: complete frame contains a torn JSONL record; there is no recovery path in the UI and the history is effectively lost from the reader's point of view.

Note: whether a given kill lands in the tolerated "structurally torn final frame" path or in this zero-tolerance branch depends on exact flush timing, so the same termination can produce either outcome.

Proposed Fix

Demote this case to the same contract already used for a structurally torn final frame, instead of throwing:

• Snapshot the scanner checkpoint before feeding the last complete frame.
• On detection, treat the offending frame as a torn tail: tornMarker.truncateTo = physical start of that frame; recoveredEvents = the complete events inside that frame (the scanner's finish() already ignores an unterminated tail record).
• Skip the best-effort recovery decompression of the region after the truncation point (anything after a known-bad frame is untrustworthy).
• Let the existing commitRepair() path truncate the artifact and re-append the recovered events plus closers. Net event loss is zero; truncation granularity is one whole frame (zstd frames are checksummed, so a mid-frame cut is not possible).We have a working local patch against 0.1.1-rc.2 implementing exactly this
  happy to share the diff in this thread if that would help.

Additional Context

• dsh: @deepseek-ai/dsh 0.1.1-rc.2 (developer preview)
• OS: Windows 10 x64 (build 10.0.18363)
• Node.js: v22.23.2
• Happy to provide a sanitized damaged session file or a minimal reproducer if useful.

[liyangbing]

这个问题与 SandBase Harness 的事件恢复目标相同，但存储实现不同：SandBase v0.3.8 的 canonical session log 是 SQLite 中的 append-only events 表，不是 DSH 的 zstd JSONL 文件，因此不能直接作为 zstd reader 修复方案。

可参考的恢复原则是：

• 每个事件有单调递增的 session-local seq，读取严格按 seq 排序；
• 事件追加不做 update/delete，恢复逻辑通过补齐未配对的 tool_use/tool_result 来让后续 projection 可继续；
• 进程启动时执行 reconcileOrphans，把中断留下的未完成工具调用转换为明确的失败结果并恢复 session 状态，而不是让整条历史不可读；
• SSE replay 只依赖已持久化的正 seq 事件，瞬时 chunk 不推进恢复游标。

实现与测试：

• https://github.com/sandbaseai/sandbase-harness/blob/main/src/core/session/event-logger.ts
• https://github.com/sandbaseai/sandbase-harness/blob/main/src/core/session/session-manager.ts
• https://github.com/sandbaseai/sandbase-harness/blob/main/tests/unit/event-logger.test.ts

对 zstd 文件格式本身，我赞同你提出的“完整 frame 内的截断尾记录按可恢复尾处理”：保留 scanner 已确认的完整记录，截断到受影响 frame 起点，并把 repair 结果写入审计信息；不要用一次尾部格式错误否定此前所有完整事件。

[ybkin1]

Follow-up to my own report above, after a deeper read of the sources. Three corrections and one honest narrowing; the overall conclusion (demote this class to the existing torn-tail contract) stands.

1. The damage mechanism in the repro needs correcting. As written, step 2 implies a non-graceful kill can directly leave the last complete frame with an unterminated final record. The writer's behavior does not support that:

• Each durable append compresses one JSONL batch into one zstd frame and writes it with a single writeFile + sync; the trailing newline is compressed into the frame payload (frames are newline-terminated by construction).
• The frame scanner parses frame structure without decompressing, so a byte-prefix cut of the final frame always scans as structurally torn — it is never reported as complete.

A plain mid-write cut therefore always lands in the tolerated class. Class (b) — a complete, checksummed frame whose plaintext tail record lacks its newline — requires a crash/persistence-timing window around a frame's write+fsync. We have no deterministic reproducer for that window; the check's existence shows the class is anticipated, and our field data (180 of 267 session logs failing with this error) shows it is the common outcome in practice. Validate with a unit test that constructs the state directly: compress a batch lacking the final newline into a complete frame, append it after a valid header frame, and assert the demotion contract (no throw; truncate at that frame's start; complete records recovered). One precision: the demotion preserves every complete record before and inside the offending frame; content after a known-bad frame is dropped by design as untrustworthy — the loss bound is per damage site, not an unconditional zero.

2. The artifact path is missing a directory level. I wrote <workspace>/<session-id>/session.jsonl.zstd. The actual layout is:

<sessions-root>/<project-key>/<encoded-session-id>/session.jsonl.zstd

The project-key level groups sessions by working directory; the session id is path-encoded.

3. Two scope claims were overstated.

• "There is no recovery path in the UI": we never examined the UI. The narrower claim: the read API hard-throws and exposes no recovery path for this class.
• "No shutdown handler that finishes or rolls back the in-flight frame": true that the module installs no process-exit hooks, but the append path does roll back to the previous size on a write error. Accurate: rollback only on write errors; no shutdown-time handling.

Environment: @deepseek-ai/dsh 0.1.1-rc.2 (developer preview), Windows 10 x64. We have applied a demotion patch in place locally and are happy to share the diff in-thread.

Thanks to the maintainers and the community. This follow-up is the result of a deeper source read before submitting the patch proposal — better exactly right than fast.