Bug: corrupt session log "seq gap in committed region" — duplicated seqs when two dsh processes write the same session (interrupted-turn closers race live resume appends)

[bpc-oss]

Bug report: corrupt session log: seq gap in committed region — duplicated seqs when two dsh processes write the same session (interrupted-turn closers race live resume appends)

Reported via GitHub Discussions per the README/CONTRIBUTING (the repository has Issues disabled).

Environment

• @deepseek-ai/dsh 0.1.0-rc.6 (npm global install; monorepo dir apps/cli)
• @deepseek-ai/dsh-session-persistence-jsonl 0.1.0-rc.6 (dir packages/session/session-persistence-jsonl)
• Node v24.15.0, Windows, session artifact session.jsonl.zstd (zstd frames, packChunks on)

Symptom

History load fails in the Web UI:

history unavailable for session "session-...": Error: corrupt session log: seq gap in committed region at line 7395 (expected 123027, got 123023)（internal）

Thrown by SessionLogScanner.consumeEventLine (packages/session/session-persistence-jsonl/lib/index.js, ~line 275-300): every stored event's seq must equal the running event count; the log is otherwise rejected (or degraded to a truncated prefix, unless a turn/end row is involved, in which case it throws).

Root cause

The JSONL backend serializes per-session-id writes only within one process (PersistenceCoordinator.serialize, packages/session/session-persistence/lib/index.js ~line 1079; appendCore ~line 829 validates event.seq === state.cursor + i). There is no cross-process coordination (no file lock, no revision compare-and-swap before append). Normal usage — the dsh web server and a CLI session both opening/resuming the same conversation — lets two processes hold independent in-memory cursors for the same log:

1. The session is interrupted mid-tool-call; the log ends with an open turn (last real event tool/call, seq N-1).
2. Process B (a load path) runs prepareCore → interruptedTurnClosers() (packages/session/dsh-session/lib/index.js line 626) and durably commits 4 synthetic closers via commitRepair: tool/result (message id interrupted-tool-result-<callId>-<seq>), step/end, turn/end (reason interrupted), session/end-seed, with seqs N..N+3.
3. Process A (a live resume path, whose cursor was seeded before those closers existed) continues appending the real continuation events starting at the same seq N.
4. Both appends pass their own in-process validation, so the artifact contains seqs N..N+3 twice.

Observed file layout (decoded from the artifact)

seq 123022 tool/call                     <- last real event before interruption
seq 123023 tool/result                   <- SYNTHETIC closer: message.id "interrupted-tool-result-call_...-123023"
seq 123024 step/end                      <- SYNTHETIC closer
seq 123025 turn/end (reason: interrupted) <- SYNTHETIC closer
seq 123026 session/end-seed              <- SYNTHETIC closer
seq 123023 tool/result                   <- REAL result for the same callId (2.7 s later)
seq 123024 step/end
seq 123025 step/start
seq 123026 assistant/chunk
...                                        (tail fully contiguous from 123027)

The identical pattern hit a second session (line 4815, expected 74251, got 74247).

Why this cannot happen in a single process

• While a session is live, prepare() throws cannot prepare session ... while it is live (dsh-session-persistence ~line 852), so closers cannot be committed over a live session.
• A stale cursor in the same process would hit append seq mismatch (~line 835) and surface as a background-write failure, not a silent duplicate.

Two independent cursors for one log therefore require two processes. (Observed: several node processes were running at corruption time; after restart only one dsh process remains, and both sessions resumed with fully contiguous appends — no recurrence in the single-process path.)

Impact

• History load hard-fails for the session.
• In our case no real conversation content was lost: the synthetic closers merely duplicate the seqs of the real rows that follow them.

Suggested fixes (any one would close this)

1. Revision compare-and-swap on append: before appending a batch, verify the persisted log still ends at the seq the in-memory cursor expects; on mismatch, re-adopt/rebase the state instead of appending blindly.
2. Cross-process file lock around the per-session append critical section (e.g., a lock file or OS-level lock).
3. Resume seeding: when a live session is created from storage whose tail contains synthetic closers, seed the live cursor from the persisted event count (closers included) so real continuation seqs can never collide with committed closers.
4. Optionally, make commitRepair/closer synthesis conditional on the log still ending at the same revision (do not commit closers over a log that has grown since it was read).

Local recovery applied (informational)

The 4 synthetic closer rows were removed per session (originals backed up); streams are strictly contiguous again (295 839 and 123 491 events), verified through the real backend read path, the Session state-machine validation, and a live session.history RPC (ok: true). Both sessions have since resumed and appended thousands of events with contiguous seqs. Happy to share the repair script or any further evidence if useful.

[bpc-oss]

Recurrence confirmed (2026-08-14, same session, same pattern).

session-8bdfce78-... hit the identical corruption again: corrupt session log: seq gap in committed region at line 24060 (expected 394173, got 394169). Decoded layout, same shape:

seq 394167 assistant/message (tool-call)            <- real
seq 394168 tool/call (pwsh "Start-Sleep -Seconds 120; ...")  <- last real event before interruption
seq 394169 tool/result (interrupted-tool-result-...) <- SYNTHETIC closer
seq 394170 step/end                                  <- SYNTHETIC closer
seq 394171 turn/end (reason: interrupted)            <- SYNTHETIC closer
seq 394172 session/end-seed                          <- SYNTHETIC closer
seq 394169 tool/result (REAL result, timestamp = tool/call + 120 290 ms)  <- real continuation
seq 394170 step/end
seq 394171 step/start
seq 394172 assistant/chunk
... (tail contiguous from 394173)

Trigger analysis for this occurrence: the agent ran a long-running tool (Start-Sleep -Seconds 120); the session was interrupted while the tool was still executing; a load path (web GUI opening the session) committed the 4 synthetic closers; 120 s later the tool actually completed and the live session (resumed in another process with a cursor from before the closers) appended the real result reusing seqs 394169-394172. The race window is "session live in process A while process B runs prepareCore/commitRepair on the same id" — the GUI opening a session that is mid-turn in another process is enough to trigger it.

Observation that may help the fix: the interrupted-tool-result closer's message.id embeds the seq assigned at generation time, and the real continuation arrives with the SAME seqs — so at read time a duplicate-seq block is fully detectable and the synthetic rows are exactly identifiable (tool/result + message.id prefix "interrupted-tool-result-" + step/end + turn/end(interrupted) + session/end-seed). A reader-side repair (drop the closer block when a real continuation with identical seqs follows) would make history load resilient even before the writer-side fix lands.

Local recovery applied again: dropped the 4 synthetic closer rows; stream contiguous 0..472726 (472 727 events); verified through the real backend read path and live session.history RPC.