llm/retry 在链状态缺失时写出重复的 retry=1 + 全新 retryId，导致会话日志被校验器判为损坏且无法打开

[maozhuoshushu]

Summary

@deepseek-ai/dsh-llm-retry 在链状态取不到时，会把同一条 policy 链上的多次重试全部写成 retry: 1，并且每次都用一个新的 retryId。这样写出的会话日志会被 @deepseek-ai/dsh-session-persistence-jsonl 的官方校验器判为损坏：

SessionFormatError: llm/retry skips its policy attempt sequence

后果是整个会话无法打开（stored session "..." is corrupt）。而且这是静默故障：损坏的日志会继续正常追加写入，直到该会话被重新打开时才暴露。

Reproduction

无法稳定复现，属低频故障。一组样本共 119 个会话，其中 1 例出现（其余 118 例通过校验）；同批另有二十余个会话存在正常递增的多重重试链（retry 1..N），所以不是「只要重试就坏」。

出问题的那一份现场特征：

• 同一 turn 内连续 27 次重试（llm/retry 27 条 + llm/retry-started 27 条），分属 9 条 policy 链（链身份 = turn / step / provider / policyKey 四元组）。
• 这 27 条 llm/retry 全部为 retry: 1，且 27 个 retryId 两两不同。按官方链规则复算：18 条违反 retry === prior.retry + 1，8 条违反「一条链只用一个 retryId」。
• 该 step 的事件区间内不存在任何 step/start 或 turn/end 事件。
• 时间线上，该会话在故障前挂起约两天后恢复运行，恢复期间发生过一次 model/selection 的 provider 切换。

Current behavior

dsh-llm-retry/lib/index.js 的 recover()：

const previous = ctx.sessionProjections.stateOf(agent.session, "llmRetry")[retryStateKey(provider, policyKey)];
const previousRetry = previous?.retry ?? 0;
if (policy.mode === "normal" && previousRetry >= policy.maxRetries) return next();
const retry = previousRetry + 1;
// retryId: previous?.retryId ?? randomUUID()

当 stateOf() 对这条链返回空表时，previousRetry 退化为 0，于是每次重试都写出 retry: 1 + 全新 UUID，与「一条链一个 retryId、retry 递增」的既定语义自相矛盾。插件本身不报错，事件照常落盘，问题被推迟到「下次打开该会话」。

已排除（仅就本样本而言）：

• 不是 apply 侧被 step/start / turn/end 清零——该 step 区间内这两个事件都不存在；
• 不是 restore 静默返回空表——dsh-session-projection/lib/index.js:296 在行不可用且 baseSeq > 0 时直接抛错，否则整表 refold。

因此嫌疑落在运行时状态：同一会话存在两个 Session 对象、projection cell 被替换、或 eager drive 未触达该 unit。这一环尚未定住。

Expected behavior

以下任选其一即可显著改善：

1. 写端自洽：stateOf() 取不到链状态时应视为不变量被破坏，而不是当作「首次重试」。至少落一条诊断事件，或直接放弃本次重试，避免写出会被自家校验器拒收的日志。
2. fail-fast：llm/retry 在 append 时若能发现自己违反链规则，应在写入前报错/给出明确诊断，而不是留下一个「沉默损坏、重开才炸」的会话。
3. 单一事实来源：写端与校验器对同一条不变量（链身份四元组 + retry 递增 + 一条链一个 retryId）应有共同定义，避免「写端不知道自己在违规」。

Environment

• @deepseek-ai/dsh: 0.1.7-alpha.2（Windows 本地安装 / 桌面版）
• @deepseek-ai/dsh-llm-retry: 0.1.7-alpha.2
• @deepseek-ai/dsh-session-persistence-jsonl: 随 0.1.7-alpha.2 安装
• 会话日志格式：v4（多帧 zstd，session.v4.jsonl.zstd）；该样本 5136 帧 / 9229 事件
• provider：一个自定义 OpenAI 兼容端点；retryPolicy 为默认值（mode: "normal", maxRetries: 5）

附：样本的修复方式（只改日志，未改写端）

对已损坏的那一份日志做了等长字符串改写（不重编码 JSON）：把违规的 18 条 retry 重编号为链内递增序号，并把 27 个 retryId 归并到链首 id。修复后官方校验器原地 PASS（events=9229），会话可正常打开。

这只是抢救了那一份文件，写端缺陷仍在，所以提这个报告。若维护者需要，可以进一步提供该样本的结构化事件序列（脱敏后）。

[argszero]

Your reading of recover() is exact, and the line numbers confirm it. Adding four things the source says that bear on which of your candidates can be the live one — the second one is the "single source of truth" item you asked for, and it explains the opposite violation direction too.

The writer's chain identity is not the validator's

Writer, packages/llm/llm-retry/src/index.ts:

// line 79
function retryStateKey(provider: string, policyKey: string): string {
  return JSON.stringify([provider, policyKey])          // two components
}
// line 131, the unit's own fold
apply: (state, event) => {
  if (event.type === 'step/start' || event.type === 'turn/end') return {}   // the ONLY reset
  ...
}

Validator, packages/session/session-format-v3-to-v4/src/relationships.ts (lines 217-221):

const prior = this.retries.findLast(item =>
  ['turn', 'step', 'provider', 'policyKey'].every(key => item[key] === data[key]))   // four components
if (attempt !== (prior === undefined ? 1 : prior['retry'] + 1))
  throw new SessionFormatError('llm/retry skips its policy attempt sequence')
if (prior === undefined ? this.retries.some(item => item['retryId'] === id) : prior['retryId'] !== id)
  throw new SessionFormatError('llm/retry must keep one retryId per policy chain')

The unit's doc comment states the two-component view as intended — "Retry state for the current step by provider and policy" (src/index.ts:118) — and it makes that true only by resetting on step/start / turn/end. The validator defines the chain as the four-tuple and never consults a reset. Two different definitions of "the chain I am on", reconciled only when the two reset boundaries happen to coincide. When they diverge:

• the fold still holds [provider, policyKey] while the four-tuple is new ⇒ the writer emits prior + 1 where the validator wants 1 — skips its policy attempt sequence without any state being lost;
• the fold has been reset (or was never populated) while the four-tuple is not new ⇒ the writer emits 1 with a fresh id — exactly the shape you captured, and the one that also trips the per-chain retryId rule (line 221: when the tuple is new the id must be unused anywhere in the session, so a fresh UUID passes; when it is not new the id must equal the prior one, so a fresh UUID fails).

That asymmetry is also why "twenty-odd sessions had normally incrementing chains and passed": there, both notions agreed.

One more thing the state cannot tell you

Line 135: if (entry?.retry === event.data.retry && entry.retryId === event.data.retryId) return state. A repeated attempt number with the same id leaves the fold unchanged — the state is a value, not a log, so it drops repeats silently. Any diagnosis built on reading this projection inherits that blind spot; the log is the only place a repeat is visible.

Where the "two Session objects" candidate sits

Your third hypothesis is the one the projection cannot rule out from inside. packages/session/session-projection/src/index.ts:

private readonly registrations = new Map<string, Registration>()
// Registration.cells: Map<Session, UnitCell>            — keyed by the INSTANCE
private cellFor(registration, session) {
  let cell = registration.cells.get(session)
  if (cell === undefined)
    cell = this.buildCell(registration.def, session.header, session.inheritedEventCount,
                          session.snapshotEvents())     // folds THIS object's log

So stateOf(agent.session, 'llmRetry') is answered per session object, not per session id. Two objects carrying one id are two independent cells, each folding only its own log — and a write that lands on the log the read is not looking at is, from the writer's side, indistinguishable from "first attempt". That is a mechanism your sample's shape is consistent with, and it is checkable without reproducing the low-frequency case: if the two Session objects are the same object, stateOf after any single append must move.

What is reachable from a plugin, and what is not

recover() and the append in backoff() are core, so your item 1 (fail at the write) and your item 2 (refuse at the append) are core-side by construction — nothing in the plugin surface can sit between the decision and session.append('llm/retry', …).

Your item 2 has a read-side analogue that is reachable, and it is the one this class of bug needs most, because the corruption is only ever discovered at the next open. We already ship that shape: @argszero/cordis-plugin-session-event-guard walks stored logs through the same validated read seam a restart uses and quotes the reader's refusal verbatim instead of predicting it, and observes session/event live so a bad write is reported at the sequence that made it — we named this failure mode deferred poison after the first report in that family (#8233). It does not currently know the llm/retry chain rule.

If it would be useful here, the same live observer can carry this invariant — four-tuple chain, retry sequence and one retryId per chain, evaluated against the events themselves rather than against the projection — so a divergence is reported in the process that wrote it, with the sequence number and both violations, instead of surfacing days later as stored session "…" is corrupt. Say the word and we will extend it and report back here either way; if the better home is core (your items 1-2 are), a read-side reporter is still the thing that tells a user which session broke and when.

[maozhuoshushu]

Verified on the preserved pre-repair copy of the affected session, using the persistence backend's own restore path rebuilt from its exported pieces (createRestore(header, {recovery:"strict", validation:"transformed"}), then V4 row admission + decode per row, then finish() + assertReleasedV4Relationships):

• all 9,229 rows are admitted — nothing else in the file is at fault — and then it throws SessionFormatError: llm/retry skips its policy attempt sequence, the same error the app hit.
• 27 llm/retry + 27 llm/retry-started in one turn across 9 steps; every llm/retry carries retry: 1 with its own retryId; one provider, one policyKey, mode: normal, maxRetries: 5.
• Replaying the reader rule gives 18 × llm/retry skips its policy attempt sequence and 18 × must keep one retryId per policy chain, both first at seq 8667 (retry=1 where that chain expects 2).

On point 1: I don't think the 2-tuple/4-tuple difference is what produces these rows. The writer's projection clears on step/start/turn/end, and the reader's chain adds turn/step — the same per-step window. With one provider and one policyKey the writer saw a single 27-long chain and the reader split it into the same 9 step windows, so inside any single window the writer's own counter should have gone 1,2,3…

What the rows show: the burst is assistant/attempt → llm/retry → llm/retry-started → assistant/attempt → …, and in 18 of the 26 repeats there is no step/start/turn/end between the previous retry of that chain and the next one (4 in one step, another 8 in a row in a later step). Those two events are the only reset in the writer's apply, so its state cannot have been cleared there — and recover() reads stateOf(agent.session, "llmRetry") while backoff() appends to that same agent.session, with those rows sitting in that session's log. The log also proves the read came back empty: serializing one of those bursts correctly requires retry up to 8 inside a single window, a value the writer's own bound forbids (previousRetry >= policy.maxRetries stops at 5). So previousRetry was read as 0 every time.

Whether that emptiness comes from a stale / re-folded cell, a different Session object carrying the same id, or a read key that no longer matches the written one, I can't tell from the log alone — the log only records what the writer believed. This session had been suspended for about two days and resumed with a provider switch shortly before the burst, which is the context where the cursor/objects get rebuilt.

One cross-check that may help: running the same restore path over the later copy of that session (same id, subsequent appends) passes with 0 violations, and across the shared 9,229 events the only rows that differ are exactly the 36 retry-family rows. So the validator is complaining about precisely these rows and nothing else.

So yes — please extend session-event-guard. The invariant worth watching is the reader's, not a key-shape one: re-derive each (turn, step, provider, policyKey) chain from the log and report the first seq where retry fails to continue the chain, or where a chain's retryId changes mid-chain. That would have flagged this file at seq 8667. The write-side fix (fail-fast, or deriving previous from the log instead of the projection) has to be core, as you said.

[argszero]

Agreed on the key-shape point, and thanks for the counter-check — it removes my earlier hypothesis for this sample: with one provider and one policyKey, the writer's reset-only-on-step/start/turn/end window and the reader's (turn, step, provider, policyKey) window cover the same span, so a 2-tuple vs 4-tuple key cannot split this chain. What actually pins previousRetry to 0 is your serialization argument: reproducing the burst inside one window would need retry to reach 8, and the writer's own previousRetry >= policy.maxRetries bound stops at 5 — so the value was read as absent, not recomputed.

Tracked: I'll extend @argszero/cordis-plugin-session-event-guard with the reader-side invariant you describe — re-derive each (turn, step, provider, policyKey) chain from the log and report the first seq where retry fails to continue the chain, and the first seq where a chain's retryId changes mid-chain — so the file would be flagged at seq 8667 instead of at load time. Two notes on scope: the audit can only see stored logs (it opens them through the same validated read seam a restart uses), so it reports the file, not the live writer; and the write-side half (fail-fast, or deriving previous from the log rather than the projection) stays core, as you said.

[xiaoshenming]

补一条和"已经写坏的会话"有关的：这条闸门拒读的已有工件可以离线修回来，不用等写入侧修好。

先复现你的形状：在一份真实的 4375 事件 v4 会话里，把同一条链（turn/step/provider/policyKey 四元组）的 llm/retry 补成三条 retry: 1、各自一个新 retryId（正是你现场的特征）：

• 官方关系折叠：llm/retry skips its policy attempt sequence —— 和你报的一字不差；
• 我们的工具当时报 ok，issues 为空 —— 这是比崩溃更糟的一类错：用户会以为文件没问题。

修法：折叠的规则本身就把两个值钉死了，所以都不是猜的 ——

• retry = 该链内的顺序（链首 1，逐次 +1）；
• retryId = 该链第一个 retryId（链内其余各条必须与它相同；折叠的第三条检查是「链首那个 id 不能已经被任何更早的 retry 用过」，所以不同链不能共用 id）；
• llm/retry-started 按 (retryId, retry) 配对，用「原配对 → 新配对」表跟着改（原配对在链内必须唯一，否则 refuse）。

实测：上面那份修完，retry 变成 1,2,3、该链只剩一个 id，assertReleasedV4Relationships 与 Session.fromRestore 两道都通过。已经合规的链一个字节都不改（fixed: 0）—— 本机 13 条真实会话扫描下来这条探测器一次都没触发，所以不会误伤健康日志。

我们 refuse 的情形（不猜）：链身份含对象、retryId 不是非空字符串、新链复用了更早出现过的 id、链内 (retryId, retry) 重复、started 配不上或坐标不一致。

边界：这只修已经落盘的工件，不碰写入侧 —— 你定位的 recover() 那条链状态丢失仍然该修（否则每次都要修一遍）。另外你提到的"写入时静默、下次打开才暴露"这点很关键：inspect 现在会直接点名 llm-retry-chain 并列出 seq，所以不必等到某次打开才发现。

安装/更新：桌面端「插件」页 → dsh-session-surgeon → 更新；repair <id> 默认 dry-run，--apply 先写 .bak.<utc>，先停写者。

[maozhuoshushu]

Thanks — I independently checked the rule you published, without executing your code.

1. The rule reproduces both released gates. I re-implemented it from your description (chain key [turn, step, provider, policyKey]; retry = position in the chain; retryId pinned to the chain's first id; llm/retry-started re-paired by its original (retryId, retry)) and ran it over a real corrupt v4 log: 18 llm/retry + 18 llm/retry-started rows rewritten, 0 refusals, 9 chains, and any single refusal aborts the whole pass exactly as you describe. The result passes row admission (9229/9229) and assertReleasedV4Relationships. It also agrees with the artifact itself: row-by-row against a log I had repaired by hand earlier, 0 rows differ.

2. Zero false positives across a local store. I scanned every session log on one machine — 126 logs, 61 of them carrying llm/retry events — and the rule fired zero times. On the already-repaired log it reports fixed: 0, so the idempotence claim holds. The only log in that set which the official reader rejects is a different class: a 410-event log refused with tool/result is outside an open turn (the #8812 prune-tail shape), where the chain rule correctly stays silent instead of reaching for it.

3. One practical obstacle for scanners. The standard plugin safety gate returns BLOCK for the repository as shipped — 15 high-severity hits. I located all of them: 14 are test-side (fake sk- literals and a fake PEM in the export/transcript tests, spawnSync in the CLI/acceptance/probe/secret tests, vm.runInNewContext loading the plugin client in two unit tests), and 1 is a false positive on pipe_to_shell that fires on the string help in the CLI entry point. The runtime paths (src/, plugin/, bin/) are clean and there is no install script — but for anyone who scans before installing, as that practice intends, the verdict is a hard stop. Moving those fixtures out of the published tree, or shipping an ignore file the scanner honours, would make the package scannable clean.

Happy to re-run the gate and confirm once a version ships that way.

[xiaoshenming]

@maozhuoshushu — thanks, the independent re-implementation is exactly the check I wanted, and a 126-log store is a far wider corpus than anything I can run here.

On point 3, one measurement that changes the picture: the published package does not contain test/, fixtures/ or scripts/. package.json's files list names bin, src, plugin, docs, skills, cordis.patch.yml, the READMEs and LICENSE, and pnpm applies that list for a github: dependency — I listed the installed copy under the desktop profile and it holds exactly those, with no test/. So the gate is scanning the repository, not what anyone installs.

That said, the two credential-shaped hits were ours to remove and now are: the fake sk-… key and the three fake PEM blocks are built at runtime (one pem(label) helper replaces the three near-identical literals), so neither sk-… nor BEGIN … PRIVATE KEY appears anywhere in the checkout any more. Test intent unchanged.

The rest I cannot remove: spawnSync in the CLI/acceptance/probe tests and vm.runInNewContext loading the plugin client are how a CLI and a browser bundle get tested at all. So the verdict stays BLOCK unless the gate skips test paths or honours an ignore file — which gate is it, and does it read one? I will ship that file if it does. Separately, the pipe_to_shell hit that fires on the string help in the CLI entry point looks like a false positive worth reporting to that tool: there is no pipe-to-shell anywhere in the repository.

One thing while you have the corpus. This release adds two report-only codes from the same walker — turn-step-mismatch (a wrong turn on turn/start, step/start, assistant/message or turn/end) and tool-result-invalid (a tool/result whose toolCallId no longer names an advertised call, disagrees with message.source.callId, or whose surfaceOp is not an object). Eight semantic mutations of a real 4376-event v4 session now report their code where we previously said ok while the released fold refused the file, and all 13 logs in my store report ok with zero hits — but 13 is a small corpus. If either code fires on any of your 126 logs, please say so; that would be a false positive I want to find before it reaches anyone else. The check is v4-only on purpose: a raw v3 tool/result keeps its call id in content[0].toolCallId with role: "user", and the v3→v4 stage lifts it into message.toolCallId, so judging the raw v3 row reports a violation the loader never sees — my first version did exactly that on one real session.

[maozhuoshushu]

把那两条码的实测口径贴一下，供后来者横向对照。我们这边独立复刻了二者（本地安装里码名还不存在，判定逻辑在 dsh-session-format-v3-to-v4 的折叠器里），对现役 39 份 v4 日志（68,885 个事件）扫描结果：tool-result-invalid 0、turn-step-mismatch 0，官方折叠全部 PASS。改用 9 个变异样本做阳性对照（4 例 turn 偏差、5 例 toolCallId/surfaceOp 变异），两条码全部命中、官方折叠全部拒读、报错子句与实现一致——所以 0 命中是「样本里没有」，不是「规则没接线」。

两个可能对别人有用的坑：

1. 拿旧版日志跑 tool-result-invalid 会得到大量假阳性：v3 把调用 id 放在 message.content[0].toolCallId 且 role="user"，v4 折叠时才上提为 message.toolCallId 并改 role。我们 8 份旧日志有 505 处命中，全部属于这一类，补上顶层字段即归零。
2. turn-step-mismatch 在我们这里唯一的真实样本，是 [Bug] 0.2.0-rc.2: tool-result pruner appends replacement tool/result outside any open turn, so the reader rejects a session its own writer produced #8812 那份 prune-tail 日志（tool/result is outside an open turn），正落在 replacement-outside-open-turn 分支——这条更像是把已知的 prune-tail 事故收紧成硬失败。

顺带更新原帖缺陷状态：39 份现役日志里 llm/retry 链 11 条全部自洽（0 变更、0 次读侧拒读），原始「重复 retry=1 + 每次新 retryId」未再现。想确认一下：这是写侧已修（stateOf(session,"llmRetry") 不再读空），还是只是没再触发 503 风暴？前者的话原帖可以关掉；后者它仍是潜伏的口子。

[xiaoshenming]

@maozhuoshushu 先回答你那个问题：写侧没修，原帖不该按已修关掉。

三版逐字节相同

我把 @deepseek-ai/dsh-llm-retry 按版本解包算了一遍 lib/index.js 的 SHA256：

0.1.7-rc.2  cbb99f1c5038e4a9167aedf333343cb3e3adfb5c9f45a37ef7562b024429d050
0.2.0-rc.2  cbb99f1c5038e4a9167aedf333343cb3e3adfb5c9f45a37ef7562b024429d050
0.2.1-rc.2  cbb99f1c5038e4a9167aedf333343cb3e3adfb5c9f45a37ef7562b024429d050

recover() 在 0.2.1-rc.2 里还是原样（lib/index.js:162）：

const previous = ctx.sessionProjections.stateOf(agent.session, "llmRetry")[retryStateKey(provider, policyKey)];
const previousRetry = previous?.retry ?? 0;
const retry = previousRetry + 1;
const retryId = previous?.retryId ?? RetryId(randomUUID());

previous 一为空，retry 就回到 1、retryId 就换新 —— 你报的两个特征（重复 retry=1 + 每次新 id）是同一个空值的两个后果，不是两个缺陷。所以「39 份里 11 条链全部自洽、原始形状未再现」只能说明没再触发，不能说明修好了。

为什么会空：两个读端的链身份不是同一个东西

• 折叠（dsh-session-format-v3-to-v4）的链身份是 (turn, step, provider, policyKey) 四元组，llm/retry 必须 = 该链上一行 + 1，且一条链只许一个 retryId。
• 这份 projection 的键是 retryStateKey(provider, policyKey)，靠 apply 的重置来承担 turn/step 那两维：

apply: (state, event) => {
  if (event.type === "step/start" || event.type === "turn/end") return {};   // 清空整张表
  if (event.type !== "llm/retry") return state;
  const key = retryStateKey(event.data.provider, event.data.policyKey);
  ...
}

两者等价的前提是重置点和折叠的 step 边界对齐。你现场那个 step 区间里没有任何 step/start / turn/end，所以重置没发生、两个读端都还在同一个 step 上 —— 那么 previous 读空只能来自投影状态本身丢了（挂起两天后恢复，投影从可见日志重建，就是这种时候）。这时它会从此一直偏离：写者自洽地继续写 retry: 1 + 新 id，而折叠按四元组找得到 prior，于是每一行都判「跳过了一次尝试」。

这解释了为什么它低频但不可逆：触发条件是「投影状态与日志不一致」，而不是「发生过重试」。要收口得让读侧不再依赖可重建的投影状态，或者把 retryStateKey 与折叠的四元组对齐；两者都不在文件里，得引擎侧改。

所以原帖建议留着：它不是已修，是潜伏。

你提的 turn-step-mismatch 命中，我这边是分开的

你说 #8812 那份 prune-tail 日志在你那边命中了 turn-step-mismatch（tool/result is outside an open turn 分支）。我核了自己这一侧，它是跳过的 —— 官方折叠对非 append 的 tool/result 在 requireTurn() 之后直接 return，从不比对它自己的 turn/step，所以我的配对判据也只对 surfaceOp === "append" 的结果行生效；「回合已关还在继续」的行归另一条（step-after-turn-end），「没有开着回合时的 replacement 结果」归 prune-tail-outside-turn。也就是说同一份日志我这边报的是一条可修码，不是 turn-step-mismatch。

如果你的实现里 replacement 行也进了 turn/step 比对，那边就是一处假阳性 —— 值得排掉，否则 prune-tail 那份日志会同时触发两条，而只有一条是它真正的原因。

关于你的 39 份 / 68,885 事件与 9 个变异样本

正是我想要的对照，谢谢 —— 0 命中 + 阳性对照全命中才说明是「样本里没有」而不是「规则没接线」。你列的两个坑我也确认：v3 的 tool/result 把调用 id 放在 message.content[0].toolCallId 且 role: "user"，v3→v4 才上提为 message.toolCallId，所以那两条码必须 v4-only（我这边就是这么收口的，注释里也写了原因，就是这个 505 处的形状）。

顺带同步一条我这边刚修掉的同类错，供你交叉检查：v0-plugin-source-form 原来不带代际地跑，于是 v3 会话里合法的 {form:"snapshot", sections} 来源被报成 v0 闸门的形状，planRepair 还给出 mustWrite: true（健康文件会被重写）。规则本身没错，错在拿阶段 N 的判据去判代际 > N 的文件。已经改为按「每条判据只在自己代际及更旧上成立」收口。