Bug: corrupt session log after unclean exit — replayed events cause seq gap in committed region, history unrecoverable

[qls-cyber]

dsh version: v0.1 developer preview (npm @deepseek-ai/dsh, installed 2026-08-14)
OS: Windows 11 (win32, Git Bash environment)
Entry point: npx @deepseek-ai/dsh web

Symptom

The Web UI refuses to load an existing session and shows:

history unavailable for session "session-074b5e84-b474-41b9-97c4-643913862e89":
Error: corrupt session log: seq gap in committed region at line 2588
(expected 46784, got 46780) (internal)

The session cannot be resumed at all.

Root cause (analyzed from the session log)

The session log (~/.dsh/sessions/<project>/session-<id>/session.jsonl.zstd, multi-frame zstd JSONL) shows the following sequence around the corruption point:

line	event type	seq
2584	tool/call	46779
2585	tool/result	46780
2586	step/end	46781
2587	turn/end	46782
2588	session/end-seed	46783
2589	tool/result	46780 ← replayed, seq goes backwards
2590	step/end	46781
2591	agent/inbox/spliced	46782
2592	step/start	46783
2593	user/message	46784

What appears to happen:

1. The session terminates (crash or unclean exit) after session/end-seed at seq 46783 is committed.
2. On the next launch, dsh re-appends events that were already committed before the crash (tool/result, step/end, ...), restarting from seq 46780 instead of 46784.
3. This creates a backward seq jump inside the committed region, and the history loader's contiguity check (expected 46784, got 46780) rejects the entire log.

Expected behavior

On restart, dsh should resume the event sequence from the last committed seq + 1 (i.e. 46784), or detect that these events are already committed and skip re-appending them. A single unclean exit should not permanently corrupt the session history.

Workaround (manual)

I was able to recover the session by decompressing the log, deleting the 4 stale replayed lines (the old copies at lines 2585-2588, keeping the newer replayed versions), and repacking the file with the same multi-frame zstd layout (frame 0 = header line only). After that the session loads normally.

Suggestion

• On startup, validate the tail of the log against the in-memory committed pointer before appending.
• Consider making the loader tolerant of a duplicated committed tail (skip events whose seq is already committed) instead of hard-failing, so users don't lose history after a crash.

Happy to provide the corrupted session.jsonl.zstd (before/after repair) privately if that helps debugging.

[argszero]

这是今天第四个独立的会话日志损坏报告——和前三个（#1333 / #1452：跨进程并发写导致 seq 段重复；#1473：损坏的首个 zstd 帧导致整个 workspace 无法启动）同属一个家族，但机制不同：你的案例是"非正常退出 → 已提交的事件被重放"（replayed committed events），而不是并发写。感谢你把行级证据表列得这么清楚。

与你行级表的对照（源码）

你的表里 seq 序列：... 46780 tool/result, 46781 step/end, 46782 turn/end, 46783 session/end-seed，然后 2589 tool/result 46780（回退重放）。关键点：

1. session/end-seed 是 resume 边界标记：packages/core/session/src/index.ts:545-546 在 resume 时若日志末尾不是 session/end-seed 会补一条。正常路径下 end-seed 之后只应跟新事件（seq 从 end-seed+1 继续）。
2. 你的日志里 end-seed（46783）之后出现了 seq 46780–46783 的重放块（tool/result/step/end/agent/inbox/spliced/step/start）——这意味着重放逻辑从某个旧游标（46780）重新开始 append，而不是从持久化的 46783+1。这确实是一个独立于并发写的机制：更像"进程崩溃时内存里未刷出的写回缓冲区（write-behind）在下次启动被重新提交，但持久化水位没有同步推进"。
3. firstLiveSeq = this.log.length（index.ts:539）——resume 后 live 日志长度直接决定新 seq；如果后端给 resume 的 seed 是截断到旧水位的（而不是文件里完整的最新水位），那么重放就会出现你看到的"end-seed 后接旧 seq"。

与 #1452 的差异确认

• BUG: 同一 DSH_HOME 下多进程并发写同一会话日志，导致 seq 重复、整段历史无法加载 #1452 = 两个进程同时写 → 文件尾部出现两段相同 seq 的并行 batch。
• 你的 = 单进程，但非正常退出后恢复路径把已提交事件当未提交重放 → 文件尾部出现"end-seed 后又一段旧 seq"。

两者都表现为"committed region 内 seq 不连续/回退"，但修复点不同：你的指向 write-behind 恢复逻辑的水位推进（恢复时应从持久化的最后一条 seq+1 继续，而不是从内存游标），#1452 指向跨进程 append 加锁。

关于你的建议

• "启动时校验 tail 与 in-memory committed pointer"：方向对，但要小心 TOCTOU（校验与 append 需原子）——我在 BUG: 同一 DSH_HOME 下多进程并发写同一会话日志，导致 seq 重复、整段历史无法加载 #1452 提过同样的坑。
• "loader 容忍重复的 committed tail（跳过已提交 seq）"：更实际。既然损坏形态是"重复段"（数据未丢、只是重复），加载器可以去重而不是硬失败——BUG: 同一 DSH_HOME 下多进程并发写同一会话日志，导致 seq 重复、整段历史无法加载 #1452 的作者也是手工删 4 行救回来的（你是删 4 行，他也是删 4 行，惊人的一致）。把"去重重复段"做成加载器的自动恢复（或 dsh session repair 工具）能一次性覆盖 BUG: 同一 DSH_HOME 下多进程并发写同一会话日志，导致 seq 重复、整段历史无法加载 #1452 + Bug: corrupt session log after unclean exit — replayed events cause seq gap in committed region, history unrecoverable #1497 两种形态。

给维护者的统一建议

今天的四个报告（#1333/#1452/#1473/#1497）已经覆盖了会话日志损坏的四种路径（并发写、重放、坏帧、emoji 中毒）。值得在 rc 阶段（SESSION_FORMAT_VERSION = 0）做一次统一的"日志完整性加固"：

1. append 前校验水位 + 跨进程锁（治 BUG: 同一 DSH_HOME 下多进程并发写同一会话日志，导致 seq 重复、整段历史无法加载 #1452）；
2. write-behind 恢复用持久化水位（治 Bug: corrupt session log after unclean exit — replayed events cause seq gap in committed region, history unrecoverable #1497）；
3. listArtifacts 逐条 skip+warn（治 [Bug] A single corrupted session log prevents the whole workspace from booting (loader entry "workspace" fails to apply) #1473 的全量停机）；
4. dsh session repair 恢复工具（自动去重重复段 / 清洗孤立代理项，覆盖 BUG: 同一 DSH_HOME 下多进程并发写同一会话日志，导致 seq 重复、整段历史无法加载 #1452/Bug: corrupt session log after unclean exit — replayed events cause seq gap in committed region, history unrecoverable #1497/bug 任何会话只要遇到截断在 emoji 中间的工具结果，就会永久 400 无法继续 #1449）。

如果需要，我可以把四个帖子的证据整理成一份完整的 feature request 草案，方便维护者直接跟进。

[Electricitysheep]

和 #1473 同族（单条损坏日志拖垮 boot）——非正常退出导致 replay events seq gap，历史不可恢复。

临时处理：#1473 的 workaround 一样适用（移走异常 session 日志再启动）；这类"日志损坏恢复"我们第 8 章会话管理记了排查路径：https://github.com/Electricitysheep/dsh-handbook/blob/main/docs/08-tools-context.md

[boyin111-1]

Thanks for the precise writeup - the backward seq jump from replayed committed events is exactly the corruption class our loader check now catches offline. I added a read-only seq-integrity scan to https://github.com/boyin111-1/dsh-doctor (check 14, runs via 'dsh-doctor --session '): it decompresses the log and walks every event's seq against its index, reporting the first gap/duplicate/rewind with line numbers and expected-vs-got values - the same rule SessionLogScanner.consumeEventLine uses to reject the log. So before opening a session you can tell whether it is corrupted and where.

It also covers the sibling reports from forced compaction leaving seq holes (#1469, which we confirmed: 1,415 holes in 88,700 lines) and concurrent writers colliding seq numbers (#1433/#1452/#1586). The scan is purely read-only - it never rewrites the log, it just tells you whether the loader will reject it and where to look.