【Bug Report】session: committed-region seq collisions from divergent next-seq cursors (stale hydrate baseline; interrupt/inbox-splice vs in-flight streamer batch) — sessions permanently unobservable; error misreports as torn record

[disheno]

session: committed-region seq collisions from divergent next-seq cursors — sessions become permanently unobservable; error misreports as "torn JSONL record"

Observed on 0.1.2-alpha.4 (Trigger A) and re-confirmed on 0.1.2-rc.1 (Trigger B; persistence package @deepseek-ai/dsh-session-persistence-jsonl 0.1.2-rc.1) / Windows 11 / Node v24.14.1, 2026-09-04.

Environment

• DSH 0.1.2-alpha.4 (Trigger A) and 0.1.2-rc.1 (Trigger B, still reproducible), Windows 11, Node v24.14.1, default JSONL persistence (zstd frames, packChunks: true)
• Two profiles (web, hindsight) sharing the global session store ~/.dsh/sessions
• Trigger B additionally involves the dsh-memory-plugin (openviking) splicing profile context (agent/inbox/spliced, target: "next-step")

Summary

SessionLogScanner.consumeEventLine requires strict seq continuity (event.seq === events.length, node_modules/@deepseek-ai/dsh-session-persistence-jsonl/lib/index.js:331-343), but nothing on the write side guarantees that every append path derives its seq from the log's real committed length. We have confirmed two independent ways to append rows whose seqs collide with already-committed events. Either way the committed region is permanently unobservable — every read path (gateway session observe, resume, verify_session) fails until the log is manually repaired — and the failure surfaces as one of two misleading messages:

failed to observe session "session-...": corrupt Zstandard session log: complete frame contains a torn JSONL record (gateway/internal)
failed to observe session "session-...": corrupt session log: seq gap in committed region at line N (expected M, got K) (gateway/internal)

Both messages are misleading: the zstd container is perfectly healthy (all frames complete and decodable, no physically torn tail — that case IS handled via tornMarker.truncateTo). The actual defect is a committed-region seq regression detected by the scanner; the first message surfaces via the checkpoint() path (committedBytes !== inputBytes, lib/index.js:1040-1041), the second via the pending-issue + turn/end throw path (lib/index.js:331-343). Which one appears depends on whether a turn/end row follows the collision.

Trigger A — stale projection-cache hydrate baseline across process restarts

1. Process A has the session open. Periodic flush order: projection-cache first, log frames second.
2. A is force-killed in the window after the cache flush but before its final log flush. Concrete evidence: cache flushed at 08:39:08.997 local, final log flush at 08:39:27.358 local — a 19 s window; cache rows record seq: 17329 while the log's true committed length is 17331 events (seq 0..17330).
3. Process B (either profile) boots and hydrates the session from the cache. Identity validation (createdAt / cwd / isSeeded / inheritedEventCount) passes — it does not compare the cached baseline against the log's real committed length.
4. The user does anything that writes from B (resume the conversation, cancel a pending inbox / steering — e.g. agent/inbox/spliced with outcome: "canceled"). B appends a row with seq = 17329, which already exists in the log.
5. The scanner sets its issue at the regression; the session becomes unobservable (message per Summary).

Trigger B — interrupt / inbox-splice closers race in-flight streamer batches (single live process, no restart)

1. A streaming turn (turn 3, step 1) is appending assistant/chunk reasoning batches. The streamer's next batch (seqs 34892..34905, 14 events) is generated and queued with its cursor at 34892.
2. Mid-stream, the user interrupts the turn. The interrupt/closer path appends, one event at a time on its own cursor: step/end seq 34892 and turn/end {reason:{kind:"interrupted"}} seq 34893 (ts T+993670), then session/end-seed seq 34894 (T+993833) and — because the memory plugin had profile context queued — agent/inbox/spliced seq 34895 (T+994518, an 11,617-byte profile payload). The log's committed next-seq is now 34896.
3. The streamer's queued batch then flushes unchanged, still carrying seqs 34892..34905 → four seqs (34892, 34893, 34894, 34895) now exist twice, written by two different paths within the same process ~850 ms apart. No process restart, no cache hydration — both cursors were live simultaneously and neither serialized against the other.
4. Every read fails at the collision (seq gap in committed region at line 4319 (expected 34894, got 34892)); the ~4,300 events / ~480 frames appended after it are unreachable until manual repair.

Evidence

Fleet scan (Trigger A, 2026-09-04)

• 46 session logs scanned with a faithful mirror of scanZstdFrames + SessionLogScanner (native node:zlib zstd, same as DSH):
  • 9 corrupt sessions, all with seq-regression signatures, 0 physically torn frames, 0 unparsable lines; every zstd frame decodes cleanly.
  • Collision times align 1:1 with process restart windows (e.g. three sessions corrupted within the same millisecond at 11:13:50 local, each exactly 2 events behind: expected 17331/got 17329, 36720/36718, 493034/493032).
  • One session was 3 events behind with 2999 subsequent frames appended on the stale baseline; another was 2 behind with 1060 frames appended (an actively-used session kept writing for ~25 minutes on the stale counter, all un-committable by readers).
  • 17 sessions without a cache record were immune (full-replay baseline is always correct); 5 further sessions carried stale cache baselines not yet written to (latent).

Single-session forensics (Trigger B, 2026-09-04, 0.1.2-rc.1)

• Log: 4,264 complete zstd frames, 4,949 JSONL lines, 39,213 events, 0 torn frames, 0 unparsable lines — container fully healthy.
• Collision zone in file order (all timestamps within one ~900 ms streaming window):
  step/end 34892 → turn/end(interrupted) 34893 → streamer batch re-using 34892..34905 (14 chunks) → session/end-seed 34894 → agent/inbox/spliced 34895 → streamer batches 34906..34925 → …
• Dup census: exactly 4 seqs written twice (34892, 34893, 34894, 34895); the other 39,205 seqs are unique. The two writers' streams interleave with no overlap beyond these 4.
• Offline repair (dedupe-only, zero data loss): dropped the streamer's 4 colliding chunk events, kept the interrupt path's real events (including the spliced profile), moved the two stray single-event frames before the trimmed batch row (file order must equal seq order), unpacked the packed row into per-event lines, re-encoded as checksummed frames. No seq renumbering (so sourceEventSeqs provenance stays byte-valid). Result: 39,209 contiguous events (0..39208); the session then loads (platform verify_session: 3 turns, 39,209 events; only the normal half-open-tail findings of an interrupted session).

Suggested fixes

1. Hydrate-time guard (projection-cache): validate the cached hydration baseline (next expected seq / event count) against the log's real committed length before trusting it; on mismatch, discard the cache record and full-replay the log (the same silent-fallback path used when the record is absent). This alone would have prevented all 9 Trigger-A corruptions.
2. Append-time guard (writer): refuse to append an event whose seq is not exactly the log's committed next-seq (fail loudly, re-observe from the log, or force full replay), instead of sealing a colliding row into a new frame. This is the only fix that covers both triggers.
3. Single seq allocator: route every append path — streamer batch pipeline, interrupt/step/turn closers, agent/inbox/spliced, cache-hydrated writers — through one serialized seq reservation, or require closer paths to rebase on the committed length at append time and join the same queue. Trigger B shows two in-process cursors diverging; reserving 34892..34905 for the queued batch would have made the closers wait or renumber.
4. Error message: distinguish a committed-region seq break from a genuinely torn record, e.g. corrupt session log: committed region breaks at seq N (expected N, got M) — likely stale hydrate baseline or concurrent writer. Today's messages sent us auditing zstd framing for nothing.

Workaround (until fixed)

• Avoid force-killing DSH processes; if a force-kill is unavoidable, stop all profiles, delete ~/.dsh/storages/session_projcache/sessions/*.json, then restart (no cache record → full replay → correct baseline; first open of each session is slower).
• Trigger B is user-reachable at any time: interrupting a streaming turn while a memory-plugin next-step splice is pending can collide the closer events with the in-flight batch. Until fixed, avoid interrupting mid-stream in sessions with the memory plugin active, or be prepared to repair.
• Recovery of already-corrupted logs: offline rewrite — decompress all frames, keep the header and every line whose seq chain continues, drop the colliding/duplicate rows (prefer keeping the interrupt/closer-path copies; they carry the real state transitions and user content), reorder so file order equals seq order, re-encode as checksummed frames (header frame must contain exactly one header line; see assertZstdHeaderFrame, lib/index.js:791-793). Do not renumber seqs — sourceEventSeqs provenance is absolute. This recipe was re-validated 2026-09-04: it restored a 39,209-event session losslessly.

Related upstream discussions: #3401, #4767. Reported by a DSH user; happy to provide the corrupted log samples and the repair scripts if useful.

[disheno]

Follow-up evidence for Trigger B: same-instant cross-session batch splice collisions (2026-09-04, ~4 h after the original report)

A re-scan of all 48 session logs on this machine found 5 newly corrupted logs within ~40 minutes, every one with the identical minimal signature: exactly one duplicated event per log, and the duplicated row is agent/inbox/spliced in all 5 cases.

session (prefix)	collided seq	splice ts (ms)	target	workspace
fe30c3db…	39213	1788529118362	next-step	D-AI-DSH
5c80c2bd…	17338	1788529118364	next-step	D-AI-DSH
de51f5ec…	45200	1788529118364	next-step	D-AI-DSH
e833c21f…	130358	1788529118364	next-step	D-AI-XXX
7f4be753…	63098	1788530938391	next-turn	D-AI-DSH

The first four collided within a 2 ms window (21:38:38.362-.364 local) across two different workspaces: the memory plugin (openviking) delivered an updated user-profile context to every session at once (agent/inbox/spliced, target: "next-step"), and each splice - appended by the closer path on its own cursor - collided with that session's in-flight streamer batch. The readers failed with both message variants (seq gap in committed region when a turn/end row follows the collision, and complete frame contains a torn JSONL record when it does not - this round surfaced mainly as the latter).

All 5 logs were repaired losslessly by dropping exactly the 1 colliding splice event per log (everything else kept byte-for-byte, no seq renumbering, so sourceEventSeqs provenance stayed valid). One of them was being actively written during the repair; the live writer's cursor converged with the repaired file and kept appending contiguously afterwards.

Takeaway: Trigger B is user-reachable at scale - a single batch delivery of the memory plugin corrupted 4 sessions across 2 workspaces in the same instant. The append-time seq guard (fix 2) or the single seq allocator (fix 3) would have prevented all 5.