Groq Qwen3-32b: reasoning missing in streaming responses with tools/tool_choice

[PClmnt]

Summary

When using LiteLLM proxy with Groq qwen/qwen3-32b, reasoning is missing in streaming responses when tools + tool_choice: auto are provided. Non‑streaming responses do include message.reasoning, and streaming without tools includes <think> tags.

Environment

• LiteLLM version: 1.81.3
• Provider: Groq
• Model: qwen/qwen3-32b
• Proxy mode (OpenAI‑compatible)

Steps to reproduce

1. Configure the model in LiteLLM with reasoning_effort: "default" in litellm_params (and drop_params: true).
2. Stream a chat completion with tools:

curl -s -H "Authorization: Bearer $LITELLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model":"<litellm_model_id>",
    "messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"test"}],
    "stream": true,
    "tools":[{"type":"function","function":{"name":"noop","description":"noop","parameters":{"type":"object","properties":{},"additionalProperties":false}}}],
    "tool_choice":"auto"
  }' \
  http://localhost:4000/v1/chat/completions

Actual

• Stream only emits content.
• No <think> tags and no reasoning deltas.
• completion_tokens_details.reasoning_tokens stays 0.

Expected

• Reasoning should be available during streaming (e.g., via reasoning_content deltas), or at least surfaced consistently with non‑streaming responses.

Comparisons

• Non‑streaming with tools returns message.reasoning for the same model.
• Streaming without tools returns <think> tags in content.

Possible cause

GroqChatCompletionStreamingHandler inherits OpenAIChatCompletionStreamingHandler and doesn’t map Groq reasoning fields into reasoning_content (OpenRouter has explicit mapping). If Groq streams reasoning separately when tools are enabled, LiteLLM may be dropping it.

[Sameerlite]

from openai import OpenAI
import os

client = OpenAI(api_key="sk-12345", base_url="http://0.0.0.0:4000")

response = client.chat.completions.create(
    **{
    "model": "qwen/qwen3-32b",
    "messages": [
        {
            "role": "user",
            "content": [{"type": "text", "text": "where is saint etienne ?"}],
        },
    ],
    "stream": True,
    "stream_options": {"include_usage": True},
    "max_tokens": 5000,
}
)

with open("response_chunks_litellm.log", "a") as log_file:
    for chunk in response:
        print(f"Chunk: {chunk} ")
        log_file.write(f"{chunk}\n")

@PClmnt thinking is coming out, I have attached the script which will show you, grok sends reasoning tokens in normal text. I compared it with normal chat completion client of openai without litellm , both send same responses in same format. Clsoing this

[PClmnt]

Thanks for checking! I think the difference is tools + tool_choice.

Your script doesn’t pass any tools, and in that case I also see <think> tags in the streamed content. But as soon as I pass tools + tool_choice: \"auto\" (which is what our app does), the stream has no <think> tags and reasoning_tokens stays 0.

from openai import OpenAI

client = OpenAI(api_key="sk-12345", base_url="http://0.0.0.0:4000")

resp = client.chat.completions.create(
    model="qwen/qwen3-32b",
    messages=[{"role":"user","content":"test"}],
    stream=True,
    stream_options={"include_usage": True},
    tools=[{
        "type": "function",
        "function": {
            "name": "noop",
            "description": "noop",
            "parameters": {"type": "object", "properties": {}, "additionalProperties": False}
        }
    }],
    tool_choice="auto",
)

for chunk in resp:
    print(chunk)

• Without tools: <think>...</think> appears in stream.
• With tools + tool_choice: auto: no <think> in stream.
• Non‑streaming responses still contain message.reasoning.

If you can reproduce the above, could you reopen? It seems specific to streaming + tools for Groq Qwen (and GPT‑OSS is the same).