Session log unreadable after resume across session/end-seed (seq gap in committed region)

[Max-Null]

Summary

A session whose log contains a session/end-seed boundary followed by a resume (retry after an interrupted turn) becomes permanently unreadable: the durable log written by DSH itself fails DSH's own scanLog validation with "corrupt session log: seq gap in committed region".

Environment

• DSH 0.1.0-rc.7 (commit 99f6f02)
• Windows; reproduced from both an Electron shell and dsh web
• Session storage: session-persistence-jsonl (zstd)

Symptom

Loading session history (web GUI history load / Session log export) fails:

corrupt session log: seq gap in committed region at line 18515 (expected 207489, got 207486) (internal)
corrupt session log: seq gap in committed region at line 5454 (expected 78435, got 78432) (internal)

The live GUI still displays the same session normally: the client conversation assembler tolerates non-contiguous seqs; only the persistence-layer strict scan rejects them.

Log shape at the gap

Two independent sessions show the identical pattern (events decoded with decodeStorageRecord):

seq=207484 step/end    t=17s=2  @17:07:30.729
seq=207485 step/start  t=17s=3  @17:07:30.772
seq=207486 step/end    t=17s=3  @17:07:30.772   (empty step - ends immediately)
seq=207487 turn/end    t=17s=?  @17:07:30.772
seq=207488 session/end-seed      @17:07:31.478   (seed marker)
seq=207486 assistant/chunk t=17s=3 @17:07:33.607  (seq goes BACKWARDS: resume restarts at seed length)
seq=207487 assistant/chunk t=17s=3 @17:07:33.608
seq=207488..207497 assistant/chunk t=17s=3       (packed row, seq0=207488)

The second session's gap is identical (expected 78435, got 78432).

This reproduced with a single DSH instance (no second instance running when the gap was written), so it is not a multi-process race.

Root cause

Session assigns seq = this.log.length (packages/core/session/src/index.ts:629). On construction with a seed (replay / resume / fork), firstLiveSeq = this.log.length (index.ts:539). When the seed's last event is not already session/end-seed, the constructor appends one (index.ts:545). After that, new events restart from firstLiveSeq (the seed length) instead of continuing after the end-seed marker.

The persistence scanner, meanwhile, requires strictly contiguous seq from 0: event.seq !== this.events.length throws "seq gap in committed region" (packages/session/session-persistence-jsonl/src/format.ts:364-371).

So any session that (1) has an interrupted/empty step (e.g. a model request failing or aborted right after step/start), (2) is then resumed (same-process retry or a later process loading it), and (3) whose resume appends events with seq restarted at the seed length, is written by DSH in a form DSH itself refuses to read later. The write side and the read side disagree about seq semantics across an end-seed boundary.

Workaround (no data loss)

Creating a fork of the session while the owning instance still holds it in memory yields a clean copy: fork seeds come from the in-memory event stream (contiguous), so the child session loads fine and the conversation continues with full context. This is the recommended recovery path for affected users. (Rewriting the log to shift seqs after the gap also works but is riskier.)

Suggested fix

Decide the intended semantics of seq across session/end-seed:

• Option A: resume continues from firstLiveSeq (current write behavior) and the scanner treats end-seed as a seq-reset boundary instead of requiring global contiguity.
• Option B: resume continues from end-seed.seq + 1 (keep global contiguity); the constructor should not append the marker at firstLiveSeq when a later resume would restart below it.

Either way, writer and strict scanner must agree, and a regression test covering "interrupted turn -> end-seed -> resume -> reload" is needed.

[argszero]

Verified against rc.7 (99f6f02fec) — this is the 12th report of the session-corruption family, and it's a new mechanism (call it #10: "resume seed shorter than the persisted tail"). The single-instance repro is the valuable part: it rules out the cross-process race (#1333/#1452) AND the stale-prepared-cache path (#2839) in one stroke, leaving a pure seed-construction arithmetic bug.

1. The exact divergence confirmed at source. Session construction (packages/core/session/src/index.ts):

• :539 this.firstLiveSeq = this.log.length — captured before the end-seed logic;
• :545 if (seed !== undefined && this.log.at(-1)?.type !== 'session/end-seed') this.append('session/end-seed', {}) — the marker is appended after firstLiveSeq was set, and the comment says it's in-memory only ("no load-time write": the backend captures the creation seed separately);
• :629 seq: this.log.length — every live append.

So within one fresh process the seqs stay in-memory contiguous (end-seed at N, next event at N+1). The divergence appears when the persisted file already contains a tail beyond the seed the constructor was given — exactly your shape: the previous session's interrupted-turn repair left turn/end + session/end-seed at 207486-207488 in the file, the new session's seed stops at 207485 (firstLiveSeq = 207486), and its first live append (seq 207486) overlaps the persisted tail → the strict scanner (format.ts:364-371, event.seq !== this.events.length → "seq gap in committed region") rejects the merged file. The step/start → step/end empty step in your trace is the interrupted turn's own footprint — consistent.

2. This is the same family shape as #2839, one level up. #2839 (resume seed excludes just-committed placeholders, via stale prepared-source cache) and #2167 (resume-path fresh-load gap) both live on the "resume seed ≠ persisted tail" axis. Yours is the fresh-load, single-instance instance of that axis — the cleanest demonstration yet that the seed boundary itself is the bug, not the cache. The coordinator's prepareCore seeds from stored events; when the seed length is computed on a view that excludes the repaired tail, every subsequent resume re-derives the same short seed and re-collides. Two sessions showing the identical pattern (expected 207489/78435, got 207486/78432) is strong evidence it's deterministic, not environmental.

3. Fix directions, in order of leverage:

• (a) Capture firstLiveSeq after the end-seed append (index.ts): move firstLiveSeq = this.log.length to after the :545 block, or recompute it there. This makes the in-memory boundary agree with the seed the backend persists. Smallest, most local fix — but it only heals new writes; existing corrupted sessions stay rejected.
• (b) Resume seed = full persisted tail (the Bug: session log corrupted on resume - last 4 committed events re-appended with fresh timestamps (seq gap in committed region) #2167/Session log corrupt after interrupted tool call + restart: seq overlap in committed region #2839 family direction): on resume, seed from the complete stored events (including the repaired tail), never from a shorter lineage view. This is the root fix — it prevents the short-seed collision at the source.
• (c) Scan-side region tolerance (format.ts): the strict scan could reset its expected counter at a session/end-seed boundary — "everything before the marker is the seed segment, everything after is live" — and enforce contiguity within each segment. This is the durable backstop (same "validate per-region" principle as the Title: [Bug] Single corrupted session.jsonl.zstd causes full DoS — no graceful degradation #2998 per-artifact quarantine): it turns a whole-file reject into a per-segment check, and it heals existing files without rewriting them.
• (d) dsh-doctor offline check: detect session/end-seed immediately followed by a seq restart → flag (and optionally truncate at the end-seed, which is the safe recovery since the seed segment is self-contained). This extends moonquake2004's existing S-checks (dsh-doctor: an offline diagnostic for the failure classes this community keeps reporting #1534) with the family's newest shape.

4. On your "GUI tolerates, persistence rejects" observation — that asymmetry is itself worth a note: the client assembler's tolerance is what makes the session look alive while exports/history fail. It's the same live-vs-durable divergence the family keeps hitting (replay-state coherence #1627/#1703/#2143); the strict scanner is the only integrity backstop, so fixing (c) must preserve strictness within each segment rather than weakening the global check.

12 reports, 10 mechanisms now — and this one comes with decoded evidence and a deterministic repro, which makes it a one-PR-sized fix ((a)+(b) together, or (a)+(c) if you want the healing path). Good find.

[Max-Null]

Thanks for the confirmation and the fix directions! I implemented (a) + (c) and verified them — the code is ready for the team to pick up:

Branch: Max-Null/deepseek-harness → fix-3198-resume-seq-gap (commit bca7700f1d), based on master (rc.7).

Fix (a) — firstLiveSeq = first seq appended AFTER construction (packages/core/session/src/index.ts):

• Capture firstLiveSeq after the constructor's session/end-seed append. It previously equaled the marker's own seq (the seed length), so an in-memory boundary disagreeing with the persisted seed was possible. The marker now always sits at firstLiveSeq - 1.
• session-persistence coordinator: the attachPrepared guard no longer compares firstLiveSeq to the cursor (the field no longer encodes the seed length); state.cursor === source.inspection.events.length remains the binding check.
• session-telemetry: adoption starts after the marker (end-seed never publishes on the firehose, so it is no longer re-exported; chunk projection unchanged).

Fix (c) — scan heals a seq break immediately after session/end-seed (packages/session/session-persistence-jsonl/src/format.ts):

• The strict scanner now treats an end-seed marker as a segment boundary: a break immediately following it re-origins the expected counter at the break, validating the live segment strictly on its own. Existing logs with overlapping segments become readable without rewriting. Breaks NOT preceded by an end-seed marker still reject as whole-file corruption (per-region strictness preserved).

Verification: 1205 tests pass across core/session, session-persistence, session-persistence-jsonl, session-telemetry, core/agent-loop, subagent-in-process-driver — including new tests for the healed restart shape, strictness inside a healed segment, and strictness of the seed segment. tsc -b tsconfig.host.json and oxlint are clean.

On (b) (resume seed = full persisted tail): the coordinator's prepareCore already seeds from the complete stored events plus synthetic closers, so the short-seed view that triggers this lives elsewhere (stale view / HMR-style adoption) — happy to chase that next, and to adjust (a)/(c) to the team's preferred shape if the maintainers want a different combination.

A git format-patch bundle is attached for convenience.

[Max-Null]

The patch bundle is here (applies cleanly on master @ rc.7): https://gist.github.com/Max-Null/16257bc912307dc15fb06e0d149305a4

[zoahdev]

⚠️ 核验提醒：这个 patch bundle 里没有修复代码——它就是官方 rc.7 的发布提交，不是 #3198 的修复。

我拉取了 gist 原文件（275KB，UTF-16）逐文件核对：

• 补丁头是 From: imccyu <cc.yu@deepseek.com>、Subject: [PATCH] release(dsh): 0.1.0-rc.7；
• 222 个改动文件全部是 package.json（版本号 2 行变更），core/session、session-persistence-jsonl 等修复目标的代码文件一个都没有；
• 描述里写的 "firstLiveSeq alignment + per-region scan healing" 实际改动不在补丁内。

所以 bca7700f1d 这个 commit 应该指向的是 rc.7 发布提交（或 git 操作时取错了哈希），@Max-Null 需要从真正的修复 commit 重新生成补丁。生成后建议用 git apply --check 在干净 master 上验证再贴（并保持 LF 编码，当前 UTF-16 会给 git 添麻烦）。

这个核验对维护者很关键：如果他们直接应用这个 bundle，只会把 222 个 package.json 版本号"打回 rc.7"而没有任何修复。

[Max-Null]

Thanks for the careful verification — you are right on all counts. The patch bundle has been regenerated.

What went wrong: the earlier bundle was exported with a wrong commit range — it captured the release(dsh): 0.1.0-rc.7 publish commit (bb4ca698d6) instead of the fix commit, and the shell redirect wrote the file as UTF-16, which is why all 222 files were package.json version bumps and no source files.

Regenerated patch (same gist, file replaced):

• Commit: bca7700f1d — fix(session): heal resume seq overlap after end-seed; align firstLiveSeq with live writes, single commit on top of 99f6f02fec (master @ rc.7)
• 9 files, +136/−33: packages/core/session/src/index.ts (firstLiveSeq captured after the end-seed append), packages/session/session-persistence-jsonl/src/format.ts (scan re-origins the expected counter at a seq break immediately after an end-seed marker), packages/session/session-persistence/src/coordinator.ts (attach guard no longer compares firstLiveSeq to the cursor), plus the affected tests
• Encoding: UTF-8, LF-only, no BOM
• Verified: git apply --check on a clean master @ 99f6f02fec passes (exit 0, all 9 files)

gist (updated): https://gist.github.com/Max-Null/16257bc912307dc15fb06e0d149305a4

Happy to adjust the (a)/(c) shape or move to (a)+(b) if the maintainers prefer a different combination.