Regression gate for the session layer — a corruption corpus that would have caught this entire bug family

[MedicineKing]

Regression gate for the session layer — a corruption corpus that would have caught this entire bug family

A proposal + first shipment of assets (corpus, verifiers, reproduction scripts). Everything in here is deliberately generic engineering practice; no implementation specifics from external systems.

The observation

This bug family has 40+ reports across ~10 distinct mechanisms — #1333, #1452, #1497, #1586, #2068, #2167, #3198, #4274, #4662 … (canonical family list, matches the #4942 proposal) — and the same readside rejection errors, in five releases, with no writer-side fix shipped yet. Each report was investigated individually, each mechanism was patched in someone's fork, and the family keeps recurring. That is not a story about individual bugs; it is a story about the absence of a checkpoint that catches the family as a class.

Let me be precise about what "caught as a class" means. Every one of these reports is a violation of exactly one contract: an event committed in the durable log must have seq === index within its committed region. The scanner enforces that contract today — but only when a user opens a session, i.e. at the worst possible moment. The existing tests are a paper tiger: each writer fix is verified in isolation, but no persistence-semantic regression gate exists — nothing replays real logs (and synthetic variants of the known shapes) through the real official read path on every change.

The proposal: three pieces, one gate

1. Deterministic corruption injector (fault injection at the event layer)

The family's shapes are few and fully enumerable:

• recovery pair: session/end-seed followed by a row with the same seq
• replayed committed prefix: N events re-appended behind a stale cursor
• overlapping synthetic closers racing a live writer (shared seqs across two write paths)
• stale-cursor continuation after seed compaction (off-by-k)
• late streaming event prose after an interrupted turn

Injection is pure text/JSONL manipulation before frame encoding, so it stays at the semantics layer: synthetic artifacts are then framed via the official frame layout (header line alone in frame 1, checksummed frames) before the referee runs — so every artifact is structurally indistinguishable from one dsh wrote itself, and the referee judges content semantics, not container quirks. That is fault injection in its classic form: known-bad input, deterministic, no entropy.

2. Corruption corpus (the first shipment)

The corpus ships in two halves, because the real artifacts are user conversations — they contain PII and are not publishable:

• Public half — deterministic synthesis: for each known shape, an injector-generated artifact with a boundary-crossing healthy case next to it (e.g. adjacent-frame valid logs, one-version-ago formats), so the corpus also guards format compatibility across versions.
• Public metadata of real cases: for every real report in this family — shape class, frame/row counts, the exact reader error string, and the observed post-fix outcome — as a table. This is what archives the family's behavior, without a byte of user content leaving the machine.

Each catalog entry carries: the observed error and the expected post-fix outcome (reader accepts, seq-contiguous from 0, history renders). New reports in the family get folded in with a one-line addition — which is how the corpus pays for itself: the next time someone finds mechanism #11, they ship it with its regression case. Private real artifacts can be contributed opt-in by affected users via a scrubbed-pipeline the maintainers approve; the public corpus never needs them.

3. Independent referee (the lesson that cost us the most)

Our first "verifier" for the repair tool was self-written — it asserted against our own understanding of the format. It was green. The repaired file then failed on the actual official reader: our rewrite produced a single zstd frame, and assertZstdHeaderFrame requires frame 1 to contain exactly one header line (first frame is not exactly one header line). Bytes were fine; structure was not; the self-written verifier could never catch it because it was judging with the same eyes that wrote the code.

So the gate's referee is the official read path itself (Session.fromRestore, assertZstdHeaderFrame, scanner contract) — not a re-implementation of it. If you fix the reader, the referee you play against tomorrow is the reader as shipped, plus the expectation list from the corpus.

Where the gate lives

A lightweight npm run verify:corpus script (or CI step) that:

1. runs the injector against the corpus shapes (and a small set of healthy logs, to prove no regression on the happy path);
2. replays every artifact through the official path and asserts the expected outcome;
3. prints a per-artifact table (same shape as the field measurement posts: reader × artifact × result) that becomes the merge-review attachment for any change touching session semantics.

Cost: minutes. Value: a change that regresses any known shape is caught before it becomes report #41.

What we're shipping (already working, zero install)

• corpus (public): deterministic synthetic artifacts for every known shape (content-free) + the real-cases metadata table (shape / frames / rows / error string / expected outcome)
• referee tool: official-path verifier (NO source modifications to dsh — it drives the published npm 0.1.1-rc.2 reader directly)
• injector: the shape generator, event-level
• one command: node verify-corpus.mjs — per-artifact table, exit code 0/1; real PII-bearing artifacts stay local and are never part of the repo

Repository: https://github.com/MedicineKing/dsh-corrupt-session-repair (MIT; the corpus pieces live alongside the repair CLI used to recover these sessions).

We'd rather this become standard upstream practice than stay a community sidecar — happy to reformat anything to fit the repo's conventions, or to run the corpus against a candidate writer-side branch (pending #4942) and publish the before/after numbers.

[Shizuku-keop]

那么alpha1版本更新后该做些什么呢bro