[Bug] 0.2.0-rc.2: tool-result pruner appends replacement tool/result outside any open turn, so the reader rejects a session its own writer produced

[satan9394]

Environment

• dsh 0.2.0-rc.2 (npm latest/next at the time of writing), Windows 11, desktop app (the web UI shows the same error).
• Session format v4 (@deepseek-ai/dsh-session SESSION_FORMAT_VERSION = 4).

Symptom

Opening a session that had been running normally fails at load:

history load failed: stored session "…" is corrupt: SessionFormatError: tool/result is outside an open turn
(raw log: %USERPROFILE%\.dsh\sessions\<project>\<session>\session.v4.jsonl.zstd)

There is no in-product way to open or repair it afterwards.

Root cause — a writer/reader mismatch in the same release

The tool-result pruner rewrites old tool results in a background/idle pass, i.e. outside any open turn. It appends, after the last turn/end:

• compaction/prune events (data.shadowedRange / data.shadowedSeqs), and
• replacement tool/result events carrying "surfaceOp": {"op":"replace","startSeq":N,"endSeq":N}.

The reader validates exactly this class in assertReleasedV4Relationships → Relationships.tool():

if (event.type === "tool/result" && event["surfaceOp"] !== "append") {
    this.requireTurn(event.type);   // throws "tool/result is outside an open turn" when turn === null
    return;
}

Since the pruner emits these after turn/end, this.turn === null, so the reader rejects a log its own writer produced.

The check runs in SessionLogScanner.finish() (dsh-session-persistence-jsonl) and is not covered by the recoverable-row path, so the whole session becomes unloadable.

This is consistent with the writer: dsh-compaction-basic's idle/manual path deliberately runs with no open turn —

if (entryState.openTurn !== null) throw new ManualCompactionError("busy", "manual compaction: the session already has an open turn");

— so a turn-outer prune pass is by design; the reader simply does not allow its output.

Evidence (single machine, 0.2.0-rc.2)

• 15 v4 sessions on disk; exactly 1 fails assertReleasedV4Relationships with this message, the other 14 pass.
• In the failing log, the last lifecycle event is turn/end (seq 3426); then 301 compaction/prune + 301 replacement tool/result events follow (seq 3427–4028), all with turn === null.
• The raw rows decode fine — the envelope {type, seq, time, data} is intact and each replacement references an existing earlier seq — so the log content itself is not damaged, only the relationship state machine rejects it.

Impact

Any long session that the pruner rewrites while idle can become permanently unloadable, with no product recovery. Related: #8153 (tool-result pruner), #6236 / #6921 (unreadable sessions), #6274 (surfaceOp eligibility).

Workaround that restored the session

Truncate the log after the last turn/end (drop the trailing compaction/prune + replacement tool/result rows). This loses no messages or turns — only the pruning metadata; the original (unpruned) tool results remain — and the log then passes assertReleasedV4Relationships. Removing the session's projection cache ($DSH_HOME/storages/session_projcache*) lets the projection rebuild.

Suggested fix (any one)

1. Treat replacement tool/result as a pure surface op that does not require an open turn, or
2. have the pruner commit its replacements inside a turn (or emit a dedicated event type) instead of a bare idle pass, or
3. add an in-product repair/quarantine for a rejected log rather than failing the whole session.

Happy to provide a redacted excerpt or a minimal synthetic log if that helps.

[satan9394]

Minimal synthetic reproduction (contents are placeholders; verified with the shipped validator assertReleasedV4Relationships from @deepseek-ai/dsh-session-format-v3-to-v4 in 0.2.0-rc.2).

A valid turn is opened and closed, then the idle pruner appends a compaction/prune and a replacement tool/result (surfaceOp:{"op":"replace",...}) after turn/end:

[{"type":"turn/start","seq":0,"time":1,"data":{"turn":1}},{"type":"step/start","seq":1,"time":1,"data":{"turn":1,"step":1}},{"type":"assistant/message","seq":2,"time":1,"data":{"turn":1,"step":1,"message":{"role":"assistant","source":{"kind":"model","provider":"p","model":"m"},"content":[{"type":"tool-call","id":"c1","name":"t","arguments":"{}"}]}},"surfaceOp":"append"},{"type":"tool/call","seq":3,"time":1,"data":{"turn":1,"step":1,"callId":"c1","name":"t","arguments":"{}"}},{"type":"tool/result","seq":4,"time":1,"data":{"turn":1,"step":1,"message":{"role":"tool","source":{"kind":"tool","callId":"c1"},"toolCallId":"c1","content":[{"type":"text","text":"original"}]}},"surfaceOp":"append"},{"type":"step/end","seq":5,"time":1,"data":{"turn":1,"step":1}},{"type":"turn/end","seq":6,"time":1,"data":{"turn":1}},{"type":"compaction/prune","seq":7,"time":1,"data":{"shadowedRange":{"start":4,"end":4},"shadowedSeqs":[4]}},{"type":"tool/result","seq":8,"time":1,"data":{"turn":1,"step":1,"message":{"role":"tool","source":{"kind":"tool","callId":"c1"},"toolCallId":"c1","content":[{"type":"text","text":"pruned"}]}},"surfaceOp":{"op":"replace","startSeq":4,"endSeq":4},"sourceEventSeqs":[4]}]

• validating the full list → FAIL: SessionFormatError: tool/result is outside an open turn
• validating it after dropping the last two rows (the turn-outer prune + replacement) → PASS

So the single defining cause is a replacement tool/result (any surfaceOp other than "append") evaluated while turn === null; Relationships.tool() calls requireTurn() and throws. No session content is needed to reproduce.

[xiaoshenming]

你那条 workaround 已经是可判定的修复了，我把它做成了工具里的一等修复，并按你的合成样例在桌面端自带的 0.2.0-rc.2 验证器上跑通了。

先说一个和 0.1.7 不同、容易看走眼的点：这段在两个发布包里不是同一份代码。

• 0.1.7-rc.2（以及 dsh-session-format-v3-to-v4 里那份）是 Relationships.tool()：if (event.type === "tool/result" && event["surfaceOp"] !== "append") { this.requireTurn(event.type); return; } → "tool/result is outside an open turn"。
• 0.2.0-rc.2 的 worker.cjs 把这段重写成了 switch，条件等价但错误串不同：case "tool/result": if (event["surfaceOp"] === "append") { requireOpenStep(...) } else if (openTurn === null) throw new SessionFormatError("tool/result replacement is outside an open turn")。
• 顺带一条能缩小探测面的：v4 的 surfaceOpOf 对 surface-eligible 类型强制要求 surfaceOp（requires a surfaceOp marker），所以「压根没有 surfaceOp」的行不可能出现在合法 v4 文件里，只认对象形态的 replace 就够。

为什么我同意这条能修、而 #7824「回合关了还在继续写」不能：replacement 是纯 surface 元数据 —— foldSurface 自己要求 sourceEventSeqs 恰好列出被遮蔽的 surface 节点，而那些节点全都还在盘上。所以裁回它跟随的那个 turn/end 就是恢复修剪前的 surface，消息、回合、工具结果一个不丢；#7824 那个位置上是真内容，裁掉就是丢数据。

我实现的判据（新 health code prune-tail-outside-turn）：

• 探测：turn/end 之后、没有开着回合时的 replacement tool/result。
• 修复：裁回它跟随的那个 turn/end，且只在「切点之后每一行都是修剪器自己的行」时执行（compaction/prune + replacement tool/result），并要求每个 sourceEventSeqs 都 ≤ 切点。
• 拒绝而不是硬修：尾巴里有别的东西（user/message、新回合、插件事件）、replacement 遮蔽到切点之后的节点、或前面根本没有 turn/end —— 三种都 refuse 并把原因写出来，因为那会变成「用一个打不开的日志换另一个打不开的日志」。这条我专门写了测试：修剪器夹在两个回合中间、后面还有真回合时，绝不动文件。
• 裁完再复查一遍兄弟探测器，确保不会留下别的拒读形状。

实测（都在本机跑过，不是转述）：

工件	官方判定
修剪器原始输出（你的合成形状）	桌面 0.2.0-rc.2 的 worker.cjs：REFUSED tool/result is outside an open turn
修复后（裁到 turn/end）	同一个 worker：VERIFIED（关系折叠 + Session.fromRestore + 内容/流比对一次跑）
健康文件	no-op（mustWrite=false）
assertReleasedV4Relationships（0.1.7-rc.2 包内）	修前拒、修后过

你提的三条建议我一条没做（那是上游的事）：读端放宽、修剪器在回合内提交、产品内建隔离/修复。我们只是在文件这一侧把用户从「只能手改 zstd 日志」里救出来。

一个提醒沿用你的结论：修完如果界面里还显示被裁掉的旧文本，要删掉那条会话的投影缓存 <home>/storages/session_projcache*/sessions/<id>.json 让它重建 —— 我只在 CHANGELOG 里提示，不代删。

要试的话：dsh plugin --profile <你的 profile> add "github:xiaoshenming/dsh-session-surgeon#main"（0.2.3），session_inspect 会先给出 prune-tail-outside-turn，session_repair 默认 dry-run、确认后 apply: true 才写盘，写前留 .bak.<utc>。

[avpanasyuk]

Scope note, checked in the published npm 0.2.0-rc.2 packages (@deepseek-ai/dsh-compaction-basic, every @deepseek-ai/* package in a dsh web install): the only references to the toolResultPruner service are the pruner itself and compaction-basic. compaction-basic calls pruneSession from exactly two places, both inside compactIfNeeded: the agent/pre-step handler (pressure) and the agent/request-error handler (context-overflow). Both run inside an open turn. Manual compaction (/compact, which requires no open turn) does not prune.

So with the npm packages, prune replacements land inside a turn and pass the validator. The turn-outer pass in the report must come from code outside those packages, presumably the desktop build named in the environment line. That narrows where the fix belongs: either the reader accepts a turn-less replace (option 1), or that caller opens a turn (option 2).