[bug] Concurrent session writers cause corrupt session log (seq collision) - history/model fail to load

[macrowanwj-droid]

Bug Report: Concurrent session writers cause corrupt session log (seq collision)

Environment

• dsh version: 0.1.1-rc.2 (installed via npm, local install)
• Node: 26.7.0
• OS: macOS 26.6.2 (arm64)
• Running under PM2 with DSH_HOME pinned to a local sandbox directory
• Model providers: MiniMax-M3 (default), DeepSeek (fallback)

Severity

High — causes the Web GUI to be unable to load session history and model list (both require reading the session log). Occurred twice in a single session on 2026-08-23. Requires manual repair of the log file (seq renumbering) to recover.

Symptom

GUI shows empty history and no models. Backend RPC returns:

corrupt session log: seq gap in committed region at line 5542 (expected 21337, got 21334)

Root Cause Analysis (two distinct corruption modes)

Mode 1: Process restart resumes writing from stale seq

1. Session is interrupted mid-turn (turn/end interrupted at turn 16)
2. Process restarts; the resumed session writes from an old seq (e.g. 21334 re-used), while committed region already reached 21336
3. Result: seq regression → log judged corrupt

Mode 2: Two Session instances concurrently write the same log (same process) ← most likely trigger

Observed corruption point:

L11147 session/end-seed seq=92744  @ 19:22:10   (resume flow appended)
L11148 agent/inbox/spliced seq=92744 @ 19:25:42  (duplicate occupancy!)

Reasoning chain:

1. A resume/recovery flow creates a new Session instance. Its constructor appends session/end-seed when the seed doesn't already end with one → consumes seq 92744
2. A background subagent report arrives (e.g. a delegated task completes). The agent writes agent/inbox/spliced through another Session instance whose in-memory log.length is still 92744 (its snapshot predates the end-seed append)
3. Two writers each use their own in-memory seq counter → same seq written twice → log corrupt

Suspected root cause

In dsh-session, the end-seed append in the Session constructor is not synchronized with the persistence cursor / in-memory log.length of other live Session instances. There is no mutual exclusion between the resume path (end-seed append) and the agent inbox injection path (agent/inbox/spliced).

The corruption is most likely to trigger when:

• A session is interrupted then resumed (creating a fresh Session instance)
• AND a background subagent report arrives during that window
• OR the process restarts and resumes writing from stale seq

Steps to reproduce (estimated)

1. Start a session with a delegated subagent task in flight
2. Interrupt the session (or trigger a resume while a subagent report is pending)
3. Both the resume flow and the subagent report inject write to the same log with overlapping seq values
4. Observe corrupt session log on next history/model load

(We did not produce a minimal repro; this is inferred from log forensics of the two incidents.)

Impact

• History loading: session.history fails
• Model loading: session.models fails (resume path reads log)
• Other sessions unaffected; backend otherwise healthy

Workaround (applied)

Manual repair: renumber all seq / seq0 / sourceEventSeqs after the conflict point to restore continuity, then re-pack the zstd multi-frame container. We also built a health-check script that scans session logs for duplicate seqs / regressions.

Suggested fix direction

• In dsh-session, make the end-seed append and the persistence cursor atomic (or re-read the latest seq from disk before appending)
• Add mutual exclusion between resume path and inbox injection path
• Validate seq allocation against the committed region before writing (fail-safe rather than silent corruption)

Additional note

max_memory_restart: '512M' in our PM2 config caused frequent restarts (DSH baseline RSS ~500MB with many plugins loaded), which increased exposure to Mode 1. We raised it to 1536M locally.

Log snippets (redacted, timestamps local)

# Mode 2 corruption point
L11147 session/end-seed seq=92744  @ 19:22:10
L11148 agent/inbox/spliced seq=92744 @ 19:25:42

# Error seen in GUI
corrupt session log: seq gap in committed region at line 5542 (expected 21337, got 21334)

[xiaoshenming]

并发写 / 恢复写者用旧 seq 追加，官方会整段拒读（seq gap in committed region）。磁盘上的 session.jsonl.zstd 不会自己好。

dsh-session-surgeon 按官方语义裁到缺口前最后一个完整 turn/end，不补缺失 seq、不重编号中间（那会发明历史）。默认 dry-run，--apply 先写 .bak.<utc>。

dsh plugin --profile web add "github:xiaoshenming/dsh-session-surgeon#main"

重启 dsh web 后左侧「会话医生」选中那条可预览。日常：会话 ⋯ → 复制会话 ID，贴进新对话接着学。

[MedicineKing]

Same family as #1497 — seq overlap/rollback in the committed region. Two zero-history-loss options (verified against the unpatched npm 0.1.1-rc.2: 931,455 events, no seq gap, Session.fromRestore passes):

1. Read-side tolerance patch: SessionLogScanner skips replayed committed rows (seq < expected) instead of refusing the whole log — everything already on disk stays, nothing gets cut. seq collision from concurrent writers — same tolerate-on-read / prevent-on-write split as above.
2. Zero-install CLI (dsh-corrupt-session-repair): node dsh-corrupt-session-repair.mjs scans (read-only) → fix previews → fix --apply (.bak.zstd first; writes official checksummed frames — header line alone in frame 1, so assertZstdHeaderFrame accepts it). Real seq holes / physical corruption are reported, never rewritten.

Pick #2 if the log is already broken; patch the readers if you want future crashes to stay invisible.