Streamed tool calls lose name/id against OpenAI-compatible servers that send explicit nulls (SGLang) — every call fails as UNKNOWN_TOOL

[BehindTheCartan]

Summary

When the runtime streams from an OpenAI-compatible server that sends explicit nulls in tool-call continuation chunks (SGLang does), the delta accumulator in packages/llm/llm-deepseek/src/translate.ts wipes the already-captured tool name and call id on every argument fragment. Every tool call then reaches the tool layer with an empty name and fails as ToolNotFoundError: unknown tool "" — with fully intact arguments. This effectively breaks agentic tool use for anyone self-hosting DeepSeek-V4-Flash behind DEEPSEEK_BASE_URL.

Environment

• deepseek-harness-sdk==0.1.0rc6 / deepseek-harness-runtime-bin==0.1.0rc6 (Linux x64)
• examples/jsonrpc-agent/minimal.cordis.yml composition, per the Python SDK guide
• DEEPSEEK_BASE_URL → SGLang OpenAI-compatible endpoint (0.0.0.dev1+g4ef1660cd) serving DeepSeek-V4-Flash
• Repro prompt: any tool-using task ("read greeting.txt, create reply.txt …")

Observed

Session JSONL, every tool call:

{"turn": 1, "step": 1, "callId": "", "name": "", "arguments": "{\"command\": \"pwd && ls -la && cat greeting.txt ...\"}"}
{"error": {"name": "ToolNotFoundError", "code": "UNKNOWN_TOOL"}}

The model retries until the step budget runs out; the task never completes.

Root cause

Captured wire chunks from SGLang for one bash call — first delta carries id + function.name, continuation deltas repeat the fields as explicit null (where the official DeepSeek API omits them):

{"delta":{"tool_calls":[{"id":"call_585b9e17c76f4bd591e549c5","index":0,"type":"function","function":{"name":"bash","arguments":""}}]}}
{"delta":{"tool_calls":[{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"{"}}]}}
{"delta":{"tool_calls":[{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"\"command\": \"pwd\"}"}}]}}

translate.ts (master @ 47f9438, lines 159–160) merges deltas with strict-undefined checks, so the explicit nulls overwrite the captured values:

if (call.id !== undefined) block.callId = call.id
if (call.function?.name !== undefined) block.name = call.function.name

After the last fragment, callId and name are null, and the tool dispatch sees "".

Suggested fix

Treat null and undefined alike when merging continuation fields, per the OpenAI streaming convention that an absent-or-null field means "no change":

if (call.id != null) block.callId = call.id
if (call.function?.name != null) block.name = call.function.name

(block.name !== undefined at line 167 has the same hazard once name has been assigned null.)

Workaround

A proxy in front of the model server that drops null id/type/function.name/function.arguments keys from streamed tool_calls deltas restores correct behavior with no other changes — confirmed end-to-end with the same composition and model.

[argszero]

Verified every claim against current master (47f9438) — the report is exact, and the fix is right. Adding the details a maintainer would want before merging:

1. Confirmation with line-level detail

• translate.ts:159-160 are exactly as cited: if (call.id !== undefined) block.callId = call.id / if (call.function?.name !== undefined) block.name = call.function.name. An explicit null from SGLang satisfies !== undefined, so it overwrites the first-delta-captured values.
• The downstream conversion you flagged is real: :71/:166 use CallId(block.callId ?? '') and :167 gates on block.name !== undefined — once name is null, the delta yield even emits name: null and the final assembly collapses it to '' (?? ''), which is exactly the "callId": "", "name": "" in your JSONL. Fixing only :159-160 makes :167 safe again by construction, but making :167 != null too is cheap defense.
• The arguments field is NOT affected: call.function?.arguments ?? '' (:161) turns a null into a no-op append, so fragments accumulate correctly — which is why your proxy workaround (dropping null id/name keys) restores full behavior with nothing else broken.

2. The regression-test gap is concrete

translate.spec.ts:105-107 covers the omission convention (DeepSeek official: continuation deltas simply omit id/name) but has no case for the explicit-null convention (SGLang: repeats the fields as null). A one-chunk addition modeled on your captured wire shape —

{ choices: [{ delta: { tool_calls: [{ index: 0, id: null, type: 'function', function: { name: null, arguments: '{"city"' } }] } }] }

— fails on current code (name/callId become empty) and passes after the != null fix. That's the minimal regression proof.

3. Scope is nicely localized

This is the only OpenAI-compatible delta translator in-tree (llm-deepseek/src/translate.ts; the pi-ai adapter uses its own streaming protocol, and there's no second OpenAI-compatible translator), so the fix is a two-line change plus one test, no cross-package blast radius. The repo's oxlint config doesn't enforce eqeqeq (.oxlintrc.json sets correctness off), so != null passes lint as-is.

4. Classification: this is the identity-loss sibling of the #1519 family

#1519 (malformed arguments replay) corrupts the payload; this corrupts the identity (id/name) while payload stays intact — same delta-assembly layer, opposite half. Both are "provider-convention divergence at the wire boundary". One asymmetry worth noting: unlike the #1337/#1593 poisoning cases, this one does NOT strand the session — the failed call gets a paired error result with the same (empty) call id, so no dangling tool_calls; it's a per-turn functional break, fully recoverable once the fix lands.

5. Suggested fix block (for when PRs reopen):

if (call.id != null) block.callId = call.id
if (call.function?.name != null) block.name = call.function.name

plus :167 → ...block.name != null ? { name: block.name } : {}, and the regression case above. Your workaround is confirmed sound as an interim measure. Excellent report — precise wire capture, correct root cause, minimal fix.