[Bug] run_code slice leaves a lone surrogate -> deepseek-official 400s every turn (0.2.0-rc.2; pi-ai routes already sanitize)

[chenyinrusi]

Duplicate-family note: this belongs to the family already reported in #1449 / #3315 / #8352 / #8466 / #8846 / #9010. I am adding an independent confirmation on desktop 0.2.0-rc.2 plus two data points I did not find above: (a) a producer path where the lone surrogate comes from model-authored run_code doing .slice(0, 120), not from a truncating DSH tool; (b) a frame-level repair that restores the original astral character instead of replacing it with U+FFFD, including the validation done before writing the session file back. If a single tracking thread is preferred, this can be folded into #8352 or #8846.

Environment

• DSH desktop 0.2.0-rc.2 (@deepseek-ai/dsh-desktop, Electron 44), Windows 11, session format v4
• provider deepseek-official, model deepseek-v4-flash, reasoningEffort: high
• @deepseek-ai/dsh-llm-deepseek 0.2.0-rc.2 (Messages transport; POST /anthropic/v1/messages)

Symptom

From one step onwards, every turn of a session fails in under a second with:

This turn failed
DeepSeek Messages request failed (400)
INVALID_REQUEST

Retry, new turns, and page reloads all fail identically. The user-visible message carries no cause.

Timeline from the session event log (v4, seq numbers)

• turn 3, step 22: model-authored run_code runs line.trim().slice(0, 120) over a grep result. The line's emoji starts at UTF-16 code unit 119, so the tool-result text ends with …）。\ud83d (event tool/result, seq 759).
• turn 3, step 23 (seq 761 → assistant/attempt seq 762): request fails ~0.9 s after step start with {"message":"DeepSeek Messages request failed (400)","code":"INVALID_REQUEST","status":400}.
• later turns (seq 772, 780) fail the same way at step 1. The poisoned event stays on the session surface, so every later request replays it.

Provider response (measured, minimal request)

POST https://api.deepseek.com/anthropic/v1/messages with the same body minus the lone surrogate → 200 text/event-stream. The same request with one unpaired \ud83d anywhere in a text block:

HTTP/1.1 400
content-type: text/plain; charset=utf-8

Failed to parse the request body as JSON: messages[0].content[0].text: unexpected end of hex escape at line 1 column 156

The body is not a JSON error envelope, so providerError() in dsh-llm-deepseek finds no error.message and falls back to the generic text; status 400 then maps to INVALID_REQUEST.

Root cause (both still present in 0.2.0-rc.2)

1. No well-formed-UTF-16 guarantee on the deepseek-official wire path. Message text is passed through verbatim and JSON.stringify emits the lone surrogate as a \udXXX escape — syntactically valid JSON, semantically invalid Unicode. This matches the cross-route asymmetry noted in HTTP 400 (INVALID_REQUEST) on every turn after a tool result contains lone UTF-16 surrogates #3315: the shipped pi-ai adapters call sanitizeSurrogates(), while the dsh-llm-deepseek Messages path has no sanitizer call site (this build's bundle contains the sanitizer only under pi-ai modules).
2. Non-JSON error bodies are discarded. !response.ok already captures the raw text, but only as cause; the surfaced LlmError keeps the fallback message. A single line of that body would have named the exact problem.

Repair path used (for sessions already poisoned)

The poisoned event is durable, so the options are to sanitize at the wire or repair the log. I repaired the log offline:

1. stop writers (confirmed no other handle: exclusive read open succeeded);
2. scan the concatenated checksummed zstd frames with the same framing rules as dsh-session-persistence-jsonl/zstd (435 frames);
3. decompress all frames, join plaintext, locate the single real \ud83d escape with a parity-aware scan (so \\ud83d text does not match); replace it with \ud83d\udd34, restoring the original character (🔴) rather than U+FFFD;
4. recompress only the affected frame with ZSTD_c_checksumFlag = 1; all other frames stay byte-identical; no seq renumbering;
5. validation before write-back: 435/435 frames decode, 784/784 lines JSON.parse, seq monotonic (last = 782), exactly 1 line differs. I could not run the in-app gates offline (assertReleasedV4Relationships / Session.fromRestore), but the file is structurally unchanged.

After an app restart the session resumes normally. Two caveats for other users: the default Fork session action is not a workaround because the omitted-atSeq default selects the latest completed-turn prefix and therefore copies the poisoned event; and replacing with U+FFFD is an equally valid repair when restoring the character is not possible.

Asks

1. Wire the existing surrogate sanitizer — or String.prototype.toWellFormed() over the payload right before JSON.stringify — into the dsh-llm-deepseek serialization boundary. Besides covering every producer (model-authored code, tool output, external logs), per the latest comment in [BUG] 工具输出含未配对 UTF-16 代理项会让整个会话永久 HTTP 400（错误信息无法诊断） #8352 this also makes already-poisoned sessions recover on the next request without touching the log.
2. When JSON.parse(text) fails for a non-2xx response, include a bounded prefix of text in the surfaced error, so unexpected end of hex escape (and nginx/413-style bodies, cf. [Bug follow-up] HTTP 413 from nginx surfaced as INVALID_REQUEST — total request-byte guard and safe session recovery #10222) stop hiding behind INVALID_REQUEST.
3. Optional defense in depth: reject or repair unpaired surrogates when a tool result is admitted to the session, before it becomes durable history.
4. If [BUG] 工具输出含未配对 UTF-16 代理项会让整个会话永久 HTTP 400（错误信息无法诊断） #8352, [Bug] read 工具按 UTF-16 码元截断会切开代理对，产生非法 JSON 并使会话永久 400 #8846, or Messages API returns bare 400 with no error body for lone UTF-16 surrogates in message content #9010 is the intended canonical thread, please point this report there — happy to fold the evidence in.

[argszero]

Your root cause 1 reproduces on 0.2.1-rc.2: there is no surrogate sanitizer call site anywhere in packages/llm. The DeepSeek Messages body is built by translate.ts, serialized once in prepareRequestExtensions (packages/llm/llm-deepseek/src/request-extensions.ts:42) and handed straight to fetch (adapter.ts:138), so an unpaired \ud83d reaches the wire as a \udXXX escape and the provider rejects the body — exactly the 400 you measured.

I am building the plugin-side repair now: a mountable package that sanitizes unpaired surrogates on the request path before serialization, so a session already poisoned by a recorded tool/result starts working again without editing its log. It repairs rather than restores — a lone surrogate carries no information about the character it was cut from, so the repaired text is U+FFFD at that position, not the original emoji you reconstructed offline — and it leaves well-formed text, including intact surrogate pairs, byte-identical.

The core fix you ask for is still yours to make and is one call: String.prototype.toWellFormed() over the payload at request-extensions.ts:42. That covers every producer at once, including the model-authored shape you found, and it is cheaper than any plugin. I will link the package in this thread when it is published, along with what it does not cover.

[argszero]

Your root cause 1 holds on the shipped artifact, and there is now a plugin that repairs the frame.

I packed @deepseek-ai/dsh-llm-deepseek@0.2.1-rc.2 and grepped it: no surrogate handling anywhere in the package — no toWellFormed, no replacement character, nothing. The body is serialized once and handed to fetch unchanged, so a string your generated code cut through a surrogate pair reaches the provider exactly as it was written, on every turn, because it is durable history.

dsh-surrogate-sanitizer-guard@0.1.0 — https://github.com/argszero/dsh-surrogate-sanitizer-guard

Install with npm i github:argszero/dsh-surrogate-sanitizer-guard. The package carries a dsh.bundle.patch, so mounting it as a bundle is enough. It listens on llm/stream, prepended: it walks the outgoing request for a string carrying an unpaired code unit, and only when it finds one does it rebuild the request with toWellFormed() — U+FFFD per unpaired unit — and re-dispatch that. A request that needs nothing reaches next() as your own object, by identity, not by a copy.

Two measurements worth passing on, because both look like a cheaper predicate and neither one works. JSON.stringify(request).isWellFormed() is always true: JSON.stringify escapes a lone surrogate into well-formed text, so the serialized form is structurally unable to witness the defect. Scanning that serialized text for escapes invents them instead — a message that literally contains the six characters of a surrogate escape, which is exactly what a run_code snippet looks like, is well formed and would be rewritten.

The boundary, stated plainly: the producer is not fixed and the durable record is not rewritten. The session log keeps the bytes your slice() wrote, so a later core fix still sees the real defect, and the fix that belongs in the harness — a sanitizer where the DeepSeek body is built — is still the harness's to make. What this does is repair the frame, so a session that is already poisoned can be used again.

Verification: 32 tests, 24 mutation arms all caught with the control running first, a mount smoke on cordis 4.0.5 with an observable teardown, and wire-level evidence that drives the published adapter with a capturing fetch — unguarded, the same field leaves as an escaped lone surrogate; guarded, it carries U+FFFD, and every other byte of the body is identical.

The earlier reports in this family (#1449 #3315 #8352 #8466 #8846 #9010) describe the same failure arriving from other producers. The frame is where they all meet.

[yamingmou]

@chenyinrusi @argszero — one boundary on "the frame is where they all meet", because it decides what a frame-level repair still catches.

Both repairs in this thread work because the damage is visible at the wire. A lone surrogate arrives as an unpaired code unit: toWellFormed() finds it, and so does a guard listening on llm/stream. The same kind of cut does not always leave something to find. When the unit being cut is a byte and the decoder is UTF-8, the cut is settled at decode time into U+FFFD — a well-formed character inside a well-formed string. At the frame nothing is unpaired and nothing is escaped, so toWellFormed() returns that payload unchanged, and the request is accepted.

Same product, different producer. subprocess-local keeps a byte window over collected output; when that window has to begin inside a character, the first retained byte is a continuation byte. finalize().text then starts �成：3 个文件 ✅ done — U+FFFD where 完 was. Nothing in the result says so: truncated: true is set for a clean cut and for this one alike, and readFrom(0).lossy is true for both, because lossy speaks about the requested offset, not about where the window begins. The two-branch isolation and the readings are in #10345, including a case that drops a whole chunk and still invents a replacement character.

So the frame is where the ill-formed producers meet, and ask 3 has the same blind spot one step earlier: by the time a tool result is admitted here, the damage is already a legal character, so a gate keyed on unpaired surrogates passes it. The two halves differ in what happens next, not in whether something was cut — one makes the following request fail, the other is carried forward as history that every layer downstream agrees with.

That is not an argument against the frame fix. It is the argument for keeping a report at the producer as well. For the ill-formed half the log still holds the evidence; for this half it holds a legal character, and the code that decided to cut is the only thing that ever knew a cut happened.

Authorship note: the readings above were taken by executing @deepseek-ai/dsh-subprocess-local@0.2.1-rc.2 — the next tag at the time of writing — as published on npm; next moved to 0.2.1-rc.3 on 2026-10-23 and its output entry is unchanged, so both give the same numbers. Drafting assisted by AI; verification and publication by me. I have no engineering background — if any technical claim reads wrong, please call it out and I will re-verify against the toolchain and correct.

Reported by the OfferKuai Team — Founder: Zhaofeng (Yaming). Website: https://www.offerkuai.com/ | Contact: contact@offerkuai.com

中文版

@chenyinrusi @argszero —— 对「帧是它们汇合的地方」这句话加一条边界，因为它决定帧级修复还能兜住什么。

这条线里的两种修法都成立，前提是损坏在线上看得见。孤立代理项送到请求里时是一个未配对的码元：toWellFormed() 找得到它，挂在 llm/stream 上的守卫也找得到。但同一类「切」并不总是留下看得见的东西。当被切的单位是字节、解码器是 UTF-8 时，那一刀在解码那一刻就被结成了 U+FFFD —— 一个良构字符串里的良构字符。到了帧上，没有未配对、没有转义，toWellFormed() 原样返回那个 payload，请求照收。

同一个产品，另一个产生方。subprocess-local 对收集到的输出维护一个字节窗口；当窗口必须从字符内部开始时，保留的第一个字节就是续字节。finalize().text 于是以 �成：3 个文件 ✅ done 打头 —— 完 的位置是 U+FFFD。结果里没有任何东西说明这一点：干净裁剪和这一例都置 truncated: true，readFrom(0).lossy 两边也都是 true，因为 lossy 说的是「你请求的偏移滑出了窗口」，不是「窗口本身从哪儿开始」。两条分支的隔离与读数都在 #10345 里，其中一例丢的是整块，照样凭空多出一个替换字符。

所以帧是非法的那些产生方汇合的地方；而 ask 3 的盲点比它早一步 —— 在这里，一个工具结果被准入的那一刻，损坏已经是合法字符了，按「未配对代理项」做的闸会放过它。这两半的区别不在「有没有被切」，而在切完之后发生了什么：一半让下一个请求报错，另一半作为历史被带下去，下游每一层都点头同意。

这不是反对帧级修复。它说的是：产生侧也必须留一份报告。对非法的那一半，日志里还留着证据；对这一半，日志里留着的是一个合法字符 —— 唯一曾经知道「这里被切过」的，是当初决定下刀的那段代码。

声明：以上读数，是把 npm 上发布的 @deepseek-ai/dsh-subprocess-local@0.2.1-rc.2（写作时的 next 标签）真正跑起来得到的；next 在 2026-10-23 移到了 0.2.1-rc.3，其 output 入口未变，两边读数一致。AI 协助起草，我负责校验与发布；我没有工程背景 —— 哪句技术表述读着不对，请直接点出来，我回到工具链重核并更正。

报告：OfferKuai（Offer快）团队 —— 创始人 赵峰（Zhaofeng / Yaming）。官网：https://www.offerkuai.com/ ｜ 联系：contact@offerkuai.com