Skip to content

v0.7.9 — fix(#364) Qwen3 thinking models empty content in non-stream mode

Choose a tag to compare

@dahai80 dahai80 released this 05 Aug 16:48
4cc214e

Fixed

#364 — Non-stream content: null on Qwen3 thinking models

Non-stream /v1/chat/completions and /v1/responses returned content: null (with completion_tokens > 0) for Qwen3 thinking models. Streaming worked; the bug was non-stream path only.

1. scheduler/spec_decode.py — the EAGLE3 draft-model speculative-decode path (spec_decode_step) finished the request without setting out.output_text on the final RequestOutput. Non-streaming reads output_text off the merged final output (engines/batched.pyclean_special_tokens), so content came back empty while tokens were still billed. Streaming (per-token new_text) was unaffected. Mirrored the output_text decode already present in the sibling ngram_spec_step and dflash_spec_step finish paths.

2. routes_internal/responses.pyenable_thinking was set as a top-level chat kwarg, but engine.chat only forwards chat_template_kwargs to the chat-template render, so the Qwen3 template ran thinking-on by default and a max_tokens-truncated response lost the visible answer. Routed enable_thinking through chat_template_kwargs and applied the shared disable-by-default (resolve_enable_thinking_default) on both stream and non-stream paths, matching /v1/chat/completions.

Test debt

  • test_responses_chat_template_kwargs.py: corrected 3 stale mock targets (vllm_mlx.service.helpers.get_configfusion_mlx.config.get_config) after the vllm_mlxfusion_mlx rename; skipped 3 TestBatchedEngineGuidedHonorsEnableThinking tests pinning a removed guided-decoding contract (tracked in #373).
  • test_stub_modules.py: removed broken TestParentWatchdogStub_parent_watchdog is now a real implementation; the old tests installed it with a bogus ppid + no on_orphan mock, so the default orphan callback SIGTERM-killed the test runner. Covered by test_parent_watchdog.py.

Related

  • #373 filed for the guided-decoding architecture drift discovered during this fix (independent latent bug).

Full diff: #374