Repository navigation
[Bug] llm-deepseek: assistant() replays a text block placed after tool_use, and strict Messages endpoints then reject the session forever #11115
meteorfall
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Affected:
@deepseek-ai/dsh-llm-deepseek0.2.1-rc.3 —packages/llm/llm-deepseek/src/serialize.ts,assistant()(~L52-82), used bydsh-tui0.17.0Severity: Medium — rare, but permanently poisons a session
Reproducibility: observed in 3 assistant messages out of 2790 tool-calling messages over 3 days; the resulting HTTP 400 is 100% repeatable once such a message is in history
Summary
assistant()preserves the model's own block order when replaying history. When an upstream reply puts its text block after itstool_useblocks — which OpenAI-compatible relays do produce — that order is written to the session and replayed verbatim. The strict Anthropic Messages endpoint then rejects every subsequent request in that session with400, permanently.Evidence
Three assistant messages in my sessions carry this shape:
{"type":"assistant/message","data":{"turn":1,"step":43,"message":{"role":"assistant","content":[ {"type":"reasoning","text":"..."}, {"type":"tool-call","id":"call_6831db58f35f4fe791143f7d","name":"pwsh","arguments":"..."}, {"type":"tool-call","id":"call_7e7e67338e28426f98480a9d","name":"pwsh","arguments":"..."}, {"type":"text","text":"⏵ 跑既有测试并核对脚本改动"} ]}}}Every
tool/callhas a matchingtool/resultin the same step, and the serializer'spendinginvariant passes — the ids are paired. What fails is the ordering requirement on the assistant side: the endpoint wants the assistant'stool_useblocks to be the last blocks of that message.Both errors are the same root cause:
The second one is the same message when its
reasoningblock is absent: with a text block trailing the tool calls, the endpoint reports the missing thinking rather than the trailing text.Isolation (what I verified)
I ran the same history against three endpoints, all with real keys:
tool_use, texttext, tool_useapi.deepseek.com/anthropic/v1/messages(native)api.deepseek.com/v1/chat/completions(OpenAI)So: the data is only rejected by the strict Messages endpoint, and only because of block order. It is not a pairing bug, not an empty-content bug, and not specific to any gateway.
I also sampled streaming replies from the relay that produces these messages. The order is not stable: in one 10-run sample, 4 replies came back
thinking, tool_use, tool_use, text(poisonous) and 6 came backthinking, text, tool_use, tool_use(fine); a later 20-run sample produced 0/20 poisonous. The upstream SSE delta order is alwaysreasoning_content → content → tool_calls, so this is the model placing its commentary inconsistently, not a conversion artifact.Proposed fix
In
assistant(), emit blocks in a stable order rather than the recorded one:thinkingblocks first, thentext, thentool_useblocks last.textbeforetool_useis accepted by the endpoint (table above), so this is safe for both routes and needs no change to the durable session format.readReplay's signature array is index-aligned tomessage.content, so the reordering has to carry the replay metadata along (zip block+replay, sort, unzip) — otherwise signatures land on the wrong block.An alternative, if preserving exact model output is preferred: keep the order but refuse to replay a poisoned history, degrading with a clear
INVALID_REPLAY_STATEdiagnostic the way cross-model history already degrades. That avoids the permanent 400 without changing what gets sent.Why this is worth fixing in the adapter
The failure mode is asymmetric: the session looks fine locally, the local invariant passes, and the damage only appears when the same session is routed to a stricter endpoint (a fallback channel, a provider switch, or a gateway whose primary channel is an OpenAI relay). By then the offending message is far in the past and every new turn fails.
Related
tool_resultcontent); the id-pairing check passing locally is common to bothrole:"tool"after text); this report is the assistant-side counterpartassistant()output-shape issueNo API keys, project contents, paths or full session logs are included. Happy to produce a patch with a regression test in
serialize.spec.tsif the maintainers want the ordering fix in this form.All reactions