About the thinking_budget.

[hereTac]

I refer to the document here thinking_budget.md. I deploy the LLM Qwen3-30B-A3B-Thinking-2507 with OpenAI-compatible APIs using the vLLM framework.

When calling this LLM's APIs for the first time, if max_tokens is preset (e.g., to 100), the model's thinking process will be truncated once it reaches approximately 100 tokens. For the second API request, even if the thinking content obtained from the first request is appended to the assistant's content as shown in the code, the LLM will still trigger a thinking process—which is an independent thinking session for the second time.

Our initial assessment suggests this issue seems to occur because continue_final_message=True in self.tokenizer.apply_chat_template is not taking effect. Does this mean that the "continue" behavior can only actually work if the underlying implementation is provided by the framework that offers OpenAI-compatible APIs (in this case, vLLM)? (Perhaps this question is more appropriately raised in the vLLM community.)

Are there any other inference frameworks you know of—such as SGLang—that already support this "continue" behavior? Additionally, when I tried launching the LLM in thinking mode via Hugging Face (HF) loading (following the Thinking Budget guide), the second request did indeed exhibit the "continue" behavior as expected.

Moreover, currently, for the second request to the OpenAI-compatible APIs, I have indirectly achieved the desired effect by adding extra_body={"chat_template_kwargs": {"enable_thinking": False}} to switch the model to non-thinking mode. I am not sure if there are any unreasonables in this approach, so please point out any issues or correct me.

[jklj077]

While we would need to review the actual code to confirm, the behavior you described is frequently caused by inadvertently calling the chat completion endpoint (/v1/chat/completions) instead of the completion endpoint (/v1/completions) for the second API request.