Qwen3.6 / vLLM: prior-turn reasoning (preserve_thinking) is stripped on replay — multi-turn thinking context lost

[jperryhouts]

Note / disclaimer: I chose to post this as an issue, rather than just submitting my own PR, because I expect that reviewing the issue report is easier than reviewing code from a random unknown contributor. I figure your agentic tools can implement the solution just as well, and I'm assuming that would actually be preferred over receiving unsolicited code for review. I used Claude Opus 4.8 to generate the content of the issue to ensure that it's as thorough and detailed as possible.

That said, I do have a patch for this already written. Let me know if you prefer for me to submit that directly.

---

Summary

When running a self-hosted Qwen3.6 model via a vLLM OpenAI-compatible endpoint (configured as a custom_providers entry), the model's chain-of-thought from earlier turns is never fed back to it on subsequent turns, even when the server is configured to preserve it. The model effectively "forgets" its own prior private reasoning.

Qwen3.6 was specifically trained to retain and leverage historical thinking traces via the preserve_thinking chat-template option (see the model card: "Qwen3.6 has been additionally trained to preserve and leverage thinking traces from historical messages…set the preserve_thinking option"). This is valuable for agent loops. Hermes currently defeats it on the client side.

Environment

• Hermes 0.17.0

• vLLM 0.23.1rc1 (OpenAI-compatible server)

• Model Qwen/Qwen3.6-35B-A3B-FP8, served with --reasoning-parser qwen3 and --enable-auto-tool-choice --tool-call-parser qwen3_coder

• Provider configured under custom_providers (base_url: http://…/v1)

• preserve_thinking enabled at the request level via chat_template_kwargs, forwarded by Hermes' per-provider extra_body config (this already works today):

  custom_providers:
    - name: my-vllm
      base_url: http://…/v1
      extra_body:
        chat_template_kwargs:
          preserve_thinking: true

  (Equivalent to the server-side --default-chat-template-kwargs '{"preserve_thinking": true}'.) Worth emphasizing: the server side is already configurable without any code change — the remaining gap is purely the client-side reasoning replay described below.

Root cause (client side)

Hermes captures and persists reasoning correctly — extract_reasoning (agent/agent_runtime_helpers.py) reads message.reasoning/reasoning_content from responses, it's stored in state.db and reloaded into history (hermes_state.py). But on replay it is stripped for any provider not in the DeepSeek/Kimi/MiMo set:

• copy_reasoning_content_for_api (agent/agent_runtime_helpers.py) pops reasoning_content unless _needs_thinking_reasoning_pad() (DeepSeek/Kimi/MiMo only) is true.
• agent/conversation_loop.py (~line 767) then unconditionally pops the internal reasoning key from the outgoing message ("reasoning field is for trajectory storage only").

So a Qwen3.6/vLLM provider receives assistant history with no thinking in any field, and the preserve_thinking template has nothing to render.

Why the existing echo-back path doesn't cover it (field-name detail)

The DeepSeek/Kimi/MiMo echo-back path replays reasoning under reasoning_content. On modern vLLM this is the wrong field:

• vLLM ≥0.20 (RFC [RFC]: reasoning_content -> reasoning vllm-project/vllm#27755) returns reasoning under reasoning, not reasoning_content.
• vLLM drops reasoning_content on incoming assistant messages ([Bug]: reasoning_content silently dropped on incoming assistant messages vllm-project/vllm#38488) — so even enabling the current echo-back for this provider would not work.

Verified empirically against vLLM 0.23.1 with the Qwen3.6 template, using the /tokenize endpoint (renders the same apply_chat_template path):

• Response body puts thinking in message.reasoning; reasoning_content is absent.

• Sending an assistant message with a reasoning_content field → token count unchanged; dumping token_strs shows the field is absent from the rendered prompt (dropped per MCP server permanently gives up after a transient backend outage; never reconnects until gateway restart #38488).

• Sending the same content under a reasoning field → renders correctly as a historical block:

  <|im_start|>assistant
  <think>
  …prior reasoning…
  </think>
  
  …answer…<|im_end|>
  

• With chat_template_kwargs: {"preserve_thinking": true} (via extra_body) the historical block renders; with preserve_thinking: false it is stripped — confirming the flag is the correct, template-agnostic server-side control and that it is respected.

Suggested fix

Add a gated notion of "this provider's template re-renders prior reasoning," and for such providers replay reasoning under the reasoning field (not reasoning_content), without the DeepSeek-style " " placeholder pad (empty <think> blocks hurt prefix caching — cf. QwenLM/Qwen3.8#131).

Concretely, three touch points:

1. A predicate (e.g. _preserves_thinking_history() in run_agent.py) identifying Qwen3.6/vLLM preserve_thinking endpoints.
2. In agent/conversation_loop.py (~767), keep a non-empty reasoning string on the outgoing message for such providers instead of popping it. (copy_reasoning_content_for_api already strips the useless reasoning_content for them.)
3. In reapply_reasoning_echo_for_provider (agent/agent_runtime_helpers.py), skip the strict-provider strip for these providers, and also strip a stale reasoning field when falling back to a strict provider.

Design note for maintainers: whether an endpoint preserves history depends on the served chat template + vLLM version, not just the model name. A config-driven switch (e.g. a preserve_thinking: true / reasoning_replay_field: reasoning flag on the custom_providers entry) would be more robust than model-name matching, and would let users opt in per endpoint. Model-name matching on qwen3.6 is a reasonable default if a config flag is undesirable.

Related

• [Bug]: reasoning_content silently dropped on incoming assistant messages vllm-project/vllm#38488 (reasoning_content dropped on input)
• [RFC]: reasoning_content -> reasoning vllm-project/vllm#27755 (reasoning_content → reasoning rename)
• chat template emits empty historical <think> blocks, causing prompt drift and cache invalidation QwenLM/Qwen3.8#131 (empty <think> blocks / prefix-cache impact)

[ggam]

This is classified as P3 (nice to have) priority. I'm not sure it can be considered a bug, but I think this deserves P2 as it can potentially improve the use of some local models dramatically.

[jperryhouts]

Anecdotally, fixing this locally made an absolutely massive difference in my confidence in my agent's ability. Remains to be seen how much the quality actually improves, but without this fix the agent would frequently get stuck in loops where it would do a lot of reasoning about a problem, then come back with a quick status update like "okay, I have everything I need. Ready to start implementing" and then it would go back into a reasoning block and have no recollection of the whole plan it just hatched, so it would have to start over. I would definitely consider that higher than P3, but obviously not my call how this gets prioritized.

[tastyrice90]

Ive noticed this issue with deepseek v4 flash on local vllm as well. The thinking is not preserved across turns.