【bug】Compaction leaves dual session files (.jsonl + .jsonl.zstd) → entire subagent subsystem fails #4746
Replies: 6 comments 1 reply
|
这个故障和 blast radius 在 rc.2 源码里可以确认，但
但 rc.2 的模型上下文
恢复时不建议让 backend 自动忽略
我把这套 rc.2 源码路径、因果边界和可逆恢复步骤补进了 Session durability runbook： |
|
Additional verified details from our environment (macOS arm64, 0.1.1-rc.2, zstd session backend): Full reproduction:
Root-cause facts (verified locally, diff-confirmed):
Workaround we used: archive the redundant Suggested fixes:
Relation to #1497: both defects sit in the session-persistence write-path invariants — #1497 is cross-process seq/watermark consistency; this one is non-atomic file-format replacement by |
|
The blast radius has a precise cause, and it is a design choice rather than an accident — which is why one session can deny the whole subsystem. Why ONE bad session breaks spawn, resume and list
const opposite = join(dir, `session${logSuffix(this.oppositeCompression())}`)
const oppositeExists = await this.exists(opposite)
if (oppositeExists) throw this.encodingMismatch(opposite)It throws out of the whole enumeration on the first directory carrying an opposite-suffix artifact. So every operation that enumerates dies with the same message regardless of which session it actually wanted — exactly your spawn / The tell: the same loop already tolerates worse damageA few lines further down, in the same walk: if (first === undefined) continue // empty/half-written file
const meta = parseHeaderMeta(first)
if (meta === undefined) continue // not a session headerAn empty log is skipped. A log whose header is unparseable is skipped. Only the duplicated artifact is fatal — and it is the mildest of the three, since your ( The fix that matches the function's own posture is to skip the inconsistent directory and report it, not to throw. That turns "one session's format inconsistency takes down ALL subagent operations" into "one session is unavailable, and here is which one" — and it costs nothing in safety, because the guard is not protecting a write here. A lead on how the pair gets createdI did not confirm the writer, but there is a shape worth checking first. // The logical artifact name is `session.jsonl` regardless of the physical
// encoding suffix (`.jsonl.zstd` marks compression only).
return { meta, filename: 'session.jsonl', content }It hands back plaintext content paired with the name On the health check"428 sessions healthy, 0 abnormal" while the subsystem is down follows from the same gap: nothing in the health path looks for an opposite-suffix sibling. If the enumeration change above lands, the skipped directories are the natural thing for a health check to report — the two fixes share one signal. |
|
Correcting my own recommendation above before anyone implements it: "skip the inconsistent directory" as I wrote it is too broad, and the existing suite catches why. Two different situations produce that suffix mismatch, and only one of them is per-session damage:
So the discriminator is whether the matching artifact also exists: if (oppositeExists) {
if (!pathExists) throw this.encodingMismatch(opposite) // wrong root — unchanged
logger.warn(`skipping ${dir} — it holds BOTH ${…} and ${…}; remove the one that does not match`)
continue
}I have this running locally with the three existing assertions unchanged and passing, plus two new ones: a store containing one healthy session and one doubled session now lists the healthy one rather than throwing, and the opposite-artifact-alone case still rejects. The first fails against current behaviour, which is the regression that matters — a test that only checks "does not throw" would also pass on an implementation that silently returned nothing.
Still unresolved, and the more important half: what writes the second file. My |
|
I implemented the both-vs-opposite discriminator described above, plus two fail-closed catalog guards, against official master b150a55.
Behavior at that exact head:
Verification: 250/250 package tests across zstd/raw, initial/cached discovery, and both project traversal orders; package typecheck and oxlint; 28/28 documentation gates; full pre-push host/client typecheck. This is blast-radius containment only. It does not establish /compact as the producer of the second file; that causal investigation remains separate. |
|
I would not make the reader silently ignore the extra file yet. That could hide the actual writer bug. On macOS I would reproduce it once with fs_usage and check which process creates session.jsonl during compact. That should confirm whether compact is really the source. |
|
A branch carrying a fix for this is available, based directly on https://github.com/nokkies/dsh-upstream-patches/tree/fix/doubled-session-files It stops one doubled session directory taking down the whole subagent subsystem. A directory holding both Offered as-is, no attribution wanted. Take, adapt, or ignore it freely. |
Uh oh!
There was an error while loading. Please reload this page.
Environment:
Summary:
After running the official
/compactcommand on sessions, some sessions end up with BOTHsession.jsonl(uncompressed) ANDsession.jsonl.zstd(compressed) files. The backend isconfigured for zstd compression; when it encounters a
.jsonlfile it fails hard with:This single session-format inconsistency breaks the entire subagent subsystem:
subagentspawn → "subagent run failed"send_message(resume) → failslist_agents(enumerate) → fails with the same errorReproduction steps:
/compactcommand on a session.Observed facts:
session.jsonlandsession.jsonl.zstd; decompressed zstdcontent was byte-identical to the .jsonl (verified via diff) — i.e. a redundant uncompressed copy
was left behind.
completely down — file-format consistency is not covered by the health checks.
Impact:
single point of failure with uncontrolled blast radius.
/compactside-effects are not validated by the read path.Workaround (what we did):
session.jsonlfiles (keeping the.zstdas the authoritative copy),Suggested fixes:
/compactshould atomically replace / clean up the uncompressed copy when producing the zstd file..jsonlfiles (or migrate them) instead of failing hard.All reactions