Check for existing issues
What happened?
When using LiteLLM to call an Ollama model (e.g., qwen3.5:9b) with tool definitions, the response from Ollama contains a valid tool_calls field, but LiteLLM does not expose it in the returned ModelResponse object. Instead, the message.tool_calls is None, and the model's text response is returned (e.g., asking for location) or, in some cases, the raw XML-style <tool_code> from the system prompt may appear.
Directly querying Ollama's API with the same payload works correctly and returns tool_calls.
Environment
OS: Windows 11
Python: 3.12.10
LiteLLM version: 1.82.4
Ollama version: 0.17.7
Model: qwen3.5:9b
Ollama's response format for tool calls includes a message.tool_calls array, each with an id, function.name, and function.arguments (as an object). This is similar to OpenAI's format, but LiteLLM may not be correctly extracting it.
The issue seems to be in LiteLLM's Ollama response adapter (litellm/llms/ollama.py), where the tool_calls from the raw response are not mapped into the ModelResponse structure.
The finish_reason should be 'tool_calls', but it remains 'stop'.
In ollama.py, when processing the response, we should check for "tool_calls" in the message and convert it to the standard ToolCall format. Currently, the code may be ignoring it or only handling function_call (deprecated). A similar issue was fixed for other providers; Ollama may need the same treatment.
Please update the Ollama integration to properly handle tool_calls in the response, so that LiteLLM users can leverage function calling with Ollama models.
Steps to Reproduce
- Install LiteLLM and Ollama, pull the model.
Run the following Python script:
python
import litellm
litellm.set_verbose = True # or set LITELLM_LOG=DEBUG
response = litellm.completion(
model="ollama_chat/qwen3.5:9b",
messages=[{"role": "user", "content": "明天天气怎么样"}],
tools=[{
"type": "function",
"function": {
"name": "get_weather",
"description": "获取天气信息",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"},
"date": {"type": "string"}
},
"required": ["location"]
}
}
}],
tool_choice="auto",
api_base="http://localhost:11434"
)
print(response)
Observe the output. The response.choices[0].message.tool_calls is None, and content contains a text response like "请问您想查询哪个城市的天气信息？" instead of triggering a tool call.
LiteLLM should parse Ollama's response and populate tool_calls in the message, allowing downstream applications (like nanobot) to invoke the tool. The finish_reason should be 'tool_calls'.
tool_calls is None.
content contains a text response, indicating the model did not return a tool call as understood by LiteLLM.
LiteLLM logs show that the request included the tools parameter correctly, but the response does not expose the tool call.
Additional Logs
Direct Ollama API call (curl)
bash
curl http://localhost:11434/api/chat -d @request.json -H "Content-Type: application/json"
json
{
"model": "qwen3.5:9b",
"messages": [{"role": "user", "content": "明天天气怎么样"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "获取天气信息",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"},
"date": {"type": "string"}
},
"required": ["location"]
}
}
}],
"stream": false
}
json
{
"model": "qwen3.5:9b",
"message": {
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_z4falxps",
"function": {
"name": "get_weather",
"arguments": {"location": "北京", "date": "明天"}
}
}
]
},
"done": true
}
- LiteLLM verbose output (from Python script)
text
SYNC kwargs[caching]: False; litellm.cache: None; kwargs.get('cache')['no-cache']: False
Final returned optional params: {'tools': [{'type': 'function', 'function': {'name': 'get_weather', 'description': '获取天气信息', 'parameters': {'type': 'object', 'properties': {'location': {'type': 'string'}, 'date': {'type': 'string'}}, 'required': ['location']}}}]}
ModelResponse(id='chatcmpl-66ffb149-b058-4fcb-8c13-bbbab00c3227', created=1773897928, model='ollama_chat/qwen3.5:9b', object='chat.completion', choices=[Choices(finish_reason='stop', index=0, message=Message(content='请问您想查询哪个城市的天气信息？请告诉我具体的城市名称，我可以帮您查询明天的天气预报。', role='assistant', tool_calls=None, function_call=None, reasoning_content='...', provider_specific_fields=None))], usage=Usage(completion_tokens=81, prompt_tokens=277, total_tokens=358, ...))
Note: The reasoning_content field appears (likely from the model), but tool_calls is absent.
Relevant log output
What part of LiteLLM is this about?
SDK (litellm Python package)
What LiteLLM version are you on ?
v1.82.4
Twitter / LinkedIn details
No response
Check for existing issues
What happened?
When using LiteLLM to call an Ollama model (e.g., qwen3.5:9b) with tool definitions, the response from Ollama contains a valid tool_calls field, but LiteLLM does not expose it in the returned ModelResponse object. Instead, the message.tool_calls is None, and the model's text response is returned (e.g., asking for location) or, in some cases, the raw XML-style <tool_code> from the system prompt may appear.
Directly querying Ollama's API with the same payload works correctly and returns tool_calls.
Ollama's response format for tool calls includes a message.tool_calls array, each with an id, function.name, and function.arguments (as an object). This is similar to OpenAI's format, but LiteLLM may not be correctly extracting it.
The issue seems to be in LiteLLM's Ollama response adapter (litellm/llms/ollama.py), where the tool_calls from the raw response are not mapped into the ModelResponse structure.
The finish_reason should be 'tool_calls', but it remains 'stop'.
In ollama.py, when processing the response, we should check for "tool_calls" in the message and convert it to the standard ToolCall format. Currently, the code may be ignoring it or only handling function_call (deprecated). A similar issue was fixed for other providers; Ollama may need the same treatment.
Please update the Ollama integration to properly handle tool_calls in the response, so that LiteLLM users can leverage function calling with Ollama models.
Steps to Reproduce
Run the following Python script:
Observe the output. The response.choices[0].message.tool_calls is None, and content contains a text response like "请问您想查询哪个城市的天气信息？" instead of triggering a tool call.
LiteLLM should parse Ollama's response and populate tool_calls in the message, allowing downstream applications (like nanobot) to invoke the tool. The finish_reason should be 'tool_calls'.
tool_calls is None.
content contains a text response, indicating the model did not return a tool call as understood by LiteLLM.
LiteLLM logs show that the request included the tools parameter correctly, but the response does not expose the tool call.
Additional Logs
Direct Ollama API call (curl)
text
Note: The reasoning_content field appears (likely from the model), but tool_calls is absent.
Relevant log output
What part of LiteLLM is this about?
SDK (litellm Python package)
What LiteLLM version are you on ?
v1.82.4
Twitter / LinkedIn details
No response