Bug: vLLM self-hosted deployments stream thinking as delta.reasoning — adapter drops it (fix available) #199
DiGuStudent
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Summary
Self-hosted DeepSeek deployments on vLLM stream thinking-mode CoT as
delta.reasoninginstead of the officialdelta.reasoning_content.dsh-llm-deepseekreads only the official field, so against such gateways every reasoning delta is silently dropped.Observed consequences (measured against a vLLM 0.25.2 deployment serving deepseek-v4-flash)
outputTokenscomes fromcompletion_tokens(which includes reasoning tokens), while the decode window only spans visible text.Wire capture
vs. the official API, which sends
delta.reasoning_content.Root cause
packages/llm/llm-deepseek/src/translate.ts:const reasoning = delta?.reasoning_content— the alias is never consulted.Fix available
Two-line fallback plus the
WireDeltafield declaration:Branch with unit tests (alias accepted, official field preferred when both arrive), README note, and an Agent Note per repo policy: https://github.com/DiGuStudent/deepseek-harness/tree/fix/vllm-reasoning-field
Verified end-to-end against the vLLM endpoint after the patch: reasoning blocks are recorded, TTFT reflects the real first token, and the decode window spans the full generation.
I know external PRs are not accepted at the moment — filing this as a bug report per CONTRIBUTING.md; happy to open a PR with this branch whenever that changes, or adjust the approach if you'd rather handle the alias elsewhere.
All reactions