INVALID_REPLAY_STATE permanently wedges a session when a pi-ai response is cut by the output cap

[SeekerCodes]

INVALID_REPLAY_STATE permanently wedges a session when a pi-ai response is cut by the output cap

Summary

When an llm-pi-ai route's assistant response is truncated at the output-token cap
(stopReason: "length") while a tool call is in flight, the persisted replay ledger
records a block that never reaches the durable message content. Every subsequent turn
in that session then fails the replay invariant with:

invalid pi-ai replay state: block count does not match assistant content
INVALID_REPLAY_STATE

The session becomes unusable through normal means — no data is lost, but the log
carries a permanently invalid ledger.

Environment

• DeepSeek Harness 0.1.0-rc.5 (local checkout; the failure was observed live on a
  running Web instance against the same log)
• Route: a hand-declared llm-pi-ai provider on the OpenAI chat-completions protocol
  pointing at the official DeepSeek endpoint
• Model: deepseek-v4-flash (response model deepseek-flash)
• No explicit maxTokens on the model, so the endpoint's own default (8192) capped the
  response

llm-pi-ai:
  providers:
    deepseek:
      apiKeyEnv: DEEPSEEK_API_KEY
      modelOverrides:
        deepseek-v4-flash:
          input: [text, image]

Steps to reproduce

1. Configure an llm-pi-ai route as above and run a session on it.
2. Ask for work that produces a long answer plus a tool call, so the answer runs into
   the output cap (here: 8192 tokens, the endpoint default when the request carries no
   explicit max_tokens).
3. The turn ends with stopReason: "length" and the truncated text; the tool call that
   was being generated never materialises in the message content.
4. Send any follow-up (e.g. "continue").
5. The run fails with INVALID_REPLAY_STATE — and keeps failing on every later turn.

Evidence

The stored assistant message (abridged) — the ledger has 2 blocks, the durable
content has 1:

{
  "type": "assistant/message",
  "data": {
    "turn": 88,
    "message": {
      "role": "assistant",
      "content": [{ "type": "text", "text": "…" }],
      "source": {
        "kind": "model",
        "provider": "deepseek",
        "model": "deepseek-v4-flash",
        "replayState": {
          "kind": "pi-ai",
          "version": 1,
          "api": "openai-completions",
          "provider": "deepseek",
          "model": "deepseek-v4-flash",
          "responseModel": "deepseek-flash",
          "responseId": "87614cec-478d-4b27-a99e-2f81fe8edf6c",
          "stopReason": "length",
          "blocks": [{ "type": "text" }, { "type": "tool-call" }]
        }
      }
    },
    "usage": { "inputTokens": 158, "outputTokens": 8192, "cacheReadTokens": 412032 }
  }
}

outputTokens: 8192 equals the endpoint default cap, and stopReason: "length" confirms
the cut. The second ledger entry (tool-call) has no counterpart in content.

A sweep of all 30 sessions in $DSH_HOME/sessions found exactly one affected session
(105 pi-ai ledger messages scanned); the single mismatch above is the whole blast radius
in that library.

Root cause

The validation that fails is intentional and correct in itself:

• packages/llm/llm-pi-ai/src/replay.ts:164

  if (state.blocks.length !== message.content.length) return invalidReplay('block count does not match assistant content')

The divergent write is the bug: toPiReplayState() builds blocks from the native
pi-ai assistant message as it completed, while the durable message persists only the
content blocks that survived the truncation. A block that was still forming when the cap
hit (a toolCall here) lands in the ledger but not in the content, so ledger and content
disagree by exactly the trailing, unpersisted blocks.

Suggested fixes (any one removes the wedge)

1. Derive, don't duplicate (preferred). Build replayState.blocks from the message
   content that is actually being persisted, so the two can never disagree. Signatures can
   be joined back by index/type.
2. Tolerate the truncation on read. When state.blocks.length > message.content.length
   and stopReason is length / aborted, align by prefix instead of throwing. This also
   lets already-broken logs recover without hand-editing, which matters because a fix to
   the write path alone leaves every existing wedged session broken.
3. Persist the partial block. Record the in-flight toolCall as a real content block
   (possibly marked incomplete) so both sides agree.

Impact

A wedged session fails every subsequent turn, so the user must abandon it or repair it
by hand. Because the trigger (hitting the output cap mid-tool-call) is reachable by
ordinary long work, the failure surfaces as an unexplained hard stop well after the
turn that actually caused it.

Workarounds (verified locally)

• Switch the session's model to another route whose adapter does not read
  replayState (e.g. llm-deepseek). The session resumes immediately; the invalid
  ledger simply goes unread. Switching back to the pi-ai route re-triggers the failure.
• Repair the stored ledger: rewrite that one message's
  source.replayState.blocks to match its content (drop the trailing, unpersisted
  block). Multi-frame zstd framing must be preserved (frame 1 = header line only).
• Prevent the trigger: declare an explicit maxTokens on the model so the request
  carries the cap instead of inheriting the endpoint default:

  llm-pi-ai:
    providers:
      deepseek:
        modelOverrides:
          deepseek-v4-flash:
            maxTokens: 65536

Notes

• packages/llm/llm-pi-ai/tests/convert.spec.ts:473 already asserts this error message,
  so the validation is by design; the uncovered path is how a ledger can diverge
  (truncated/aborted streams), not the check itself.
• A read-side tolerance (fix 2) is worth having even alongside fix 1, since it is the only
  way existing affected logs recover without manual surgery.