Session rows after a seq gap are silently dropped — only surfaces if a turn/end happens to follow

[FengYun-66659991]

Hi, reporting a data-loss issue in the JSONL session persistence (found on @deepseek-ai/dsh-session-persistence-jsonl@0.1.2-rc.1, and we re-checked it against 0.1.5-rc.2).

What happens. While loading a session log, if an event row has a seq that doesn't equal the number of events loaded so far, the loader records the error and silently keeps only the contiguous prefix, dropping everything after it — unless a turn/end row appears later in the file, in which case it throws.

Why it's dangerous. So the same corruption either fails loudly or loses history silently, depending on whether a turn/end happens to follow:

no turn/end after the gap → the session opens normally; part of the history is just gone (no error, no warning);
a turn/end after the gap → hard error, session won't load at all.
In our case a real session (14,871 rows / 380,947 events) became unloadable this way, and we found 3 leftover corrupt_session_* directories in the same environment — so it's recurring, not a one-off.

Where (0.1.2-rc.1) — lib/index.js, consumeEventLine():

if (event.seq !== this.events.length) {
const expected = this.events.length;
this.events.length = rowStart; // roll back this row
this.issue = new Error(corrupt session log: seq gap ... (expected ${expected}, got ${event.seq}));
if (decoded.some(c => c.type === "turn/end")) throw this.issue; // loud only if turn/end follows
return; // otherwise: silent
}
The error object is already constructed — it just has no observable outlet on the silent path.

We also checked 0.1.5-rc.2. The recovery policy is now a parameter (createDecoder(value, recovery)), and strict is used — but only on the verify/migration path:

decodeCurrentGeneration() → new SessionLogScanner(..., "strict") → throws loudly;
the actual session read path still defaults to recoverable:
readStoredLog → decodeStoredLog → readZstdPrefix() → new SessionLogScanner(headerFrame.value) ← no recovery argument, i.e. the constructor default "recoverable".
So the silent-drop behavior is still reachable on the read path in 0.1.5-rc.2.

Suggested fix — either is fine by us:

(a) keep the read path tolerant on purpose → then at least surface the already-constructed issue somewhere observable (log line / session health / UI notice). We're not asking for a behavior change, only for it not to be silent.
(b) the read path shouldn't be tolerant → pass "strict" to the scanner in readZstdPrefix(); the same default is already overridden two lines apart in the same file.
Repro material we can provide on request: the pre-fix log (6,211,075 B, SHA256 3E39AAA6…), a read-path replica validator, and a renumbering tool with backup+re-validate+auto-rollback built in.

Happy to file a fuller write-up or test a patch if useful. Thanks!

— 超脑团队 (Super Brain team)

[FengYun-66659991]

Correction — prior art: we are not claiming to be first

Following up on this post, we searched the existing discussions and found #3631 — "History read path silently treats a valid aborted-turn session log as 'no more history' — root cause + fix path" (#3631) — which already reports the same read-path behaviour and proposes a root cause and a fix path. Credit to that report: we are not claiming to be first here, and this post should be read as an additional data point rather than a new finding. (A search also surfaced #4795 and several other seq-related session-log reports; we have not verified those individually and are not citing them as evidence.)

What we believe is still additive:

1. A bit-for-bit re-check of the latest release, 0.1.5-rc.2 (fetched per the registry dist.integrity and sha512-verified). The recovery policy is now a parameter and "strict" is used — but only on the verify/migration path (decodeCurrentGeneration(), L1853/L1867). The actual session read path still defaults to "recoverable": readStoredLog → decodeStoredLog → readZstdPrefix() → new SessionLogScanner(headerFrame.value) (L2807, no recovery argument). So the silent-drop behaviour is still reachable on the read path in 0.1.5-rc.2.2. A real damaged sample: the pre-fix log, 6,211,075 B, SHA256 3E39AAA6… (full hash and the sample itself available on request).3. A 7-case tool matrix that we actually executed — syntax / healthy log / damaged log with correct offset / damaged log with wrong offset / usage error / normal apply / injected verification failure → automatic rollback — plus a read-path replica validator.4. Either fix is fine by us: (a) the read path is intentionally tolerant → then at least surface the already-constructed issue somewhere observable (log line / session health / UI notice); or (b) it should not be tolerant → pass "strict" in readZstdPrefix() — the same default is overridden two lines apart in the same file.

Thanks to the author of #3631 — our material is available as corroborating evidence there if useful.

— 超脑团队 (Super Brain team)

[PerryLink]

Confirmed on master (c291e79) with line numbers — the silent drop is structural, and the error is constructed before it gets swallowed:

• The history read path instantiates SessionLogScanner without a recovery argument (packages/session/session-persistence-jsonl/src/index.ts:915); the constructor defaults to recoverable (src/format.ts:401-410).
• The gap IS detected: row decode runs the v1→v2 codec's seq-gap check (packages/session/session-format-v1-to-v2/src/codec.ts:100-112). Under recoverable it parks the error and silently drops the rest of the prefix (src/format.ts:497-515); only a later committed turn/end rethrows it.
• The header-side restore is even created with recovery: 'strict' (src/format.ts:366-371), so the refusal exists by construction — but the scanner's own default mode governs. Strict is only used on the verification path (src/generation.ts:581/595), and the recoverable semantics are pinned by tests (tests/jsonl.spec.ts:2163/2175/2225).

Minimal fix: pass 'strict' at index.ts:915 (turns silent truncation into a read error), or expose the parked issue (format.ts:394) so callers can surface "history ends at seq N, dropped rows follow". That is exactly the "valid aborted-turn session treated as no more history" class you and #3631 reported.

(I verified the persistence side only; the client-side half of the chain is outside what I checked.)