[Bug]: Function/Tool Calling Returns Raw JSON Instead of Executing Tools (Ollama via LiteLLM)

[rcmurphy]

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

Summary

When using LiteLLM as a proxy between OpenWebUI and Ollama, function/tool calls are returned as raw JSON in the message content instead of being properly structured as tool calls. The same tools work correctly when routed through OpenRouter (e.g., with Claude models), and also work correctly when OpenWebUI connects directly to Ollama (bypassing LiteLLM).

Expected Behavior

When the model decides to use a tool, the response should contain a properly structured tool_calls array, and the client (OpenWebUI) should execute the tool call.

Actual Behavior

The model's tool call is returned as raw JSON in the message content field and displayed to the user instead of being executed:

{"name": "search_knowledge_files", "arguments": {"query": "fun facts about the Roman Empire"}}

Possible Causes

Speculating here!

1. Response parsing issue: Ollama returns a proper tool call structure, but LiteLLM fails to parse it correctly and passes the raw JSON through as message content.

2. Request formatting issue: LiteLLM sends the tool definitions to Ollama in a format that causes Ollama to respond with JSON in the content field rather than using its native tool call response structure.

Additional Information

Further diagnostic information (including proxy logs showing raw request/response data) is available on request.

This appears to be related to previously reported issues (#11104, #11262, #17807) that were either incompletely fixed or have regressed.

Steps to Reproduce

Configuration

Model string passed to LiteLLM: ollama/hf.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF:Q4_K_M

Native function calling is enabled. Multiple permutations of function calling-related settings (eg. supports_function_calling) have been tested with no change in behavior. I have also tried with and without a manual tool_choice. The models are configured to stream in LiteLLM.

Instructions

1. Configure LiteLLM proxy with an Ollama model that supports tool calling (e.g., Qwen3 Instruct)
2. Configure OpenWebUI to use LiteLLM as the backend
3. Enable native function calling in OpenWebUI
4. Send a message that should trigger a built-in tool call (e.g., "Search for fun facts about the Roman Empire")
5. Observe that instead of executing the tool, the raw JSON is displayed in the chat

Relevant log output

N/A

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

v1.81.0

Twitter / LinkedIn details

No response

[rcmurphy]

Response from Claude:

{
  "id": "gen-1769527383-fYVu3OG3bLjAU6h0uIDy",
  "model": "anthropic/claude-haiku-4.5",
  "usage": {
    "total_tokens": 86,
    "prompt_tokens": 16,
    "completion_tokens": 70,
    "prompt_tokens_details": null,
    "completion_tokens_details": {
      "text_tokens": null,
      "audio_tokens": null,
      "image_tokens": null,
      "reasoning_tokens": 0,
      "accepted_prediction_tokens": null,
      "rejected_prediction_tokens": null
    }
  },
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "I'll search for some interesting information about the Roman Empire for you.",
        "tool_calls": [
          {
            "id": "toolu_vrtx_019KdvGgYTMjEn9ShjpRvFrP",
            "type": "function",
            "function": {
              "name": "search_web",
              "arguments": "{\"query\": \"fun facts about roman empire\", \"count\": 5}"
            }
          }
        ],
        "function_call": null
      },
      "finish_reason": "tool_calls"
    }
  ],
  "created": 1769527384,
  "system_fingerprint": null
}

Response from Ollama:

{
  "id": "chatcmpl-12faac0e-9675-48ec-808e-2744290abd4f",
  "model": "hf.co/unsloth/Qwen3-30B-A3B-Thinking-2507-GGUF:Q4_K_M",
  "usage": {
    "total_tokens": 3519,
    "prompt_tokens": 3496,
    "completion_tokens": 23,
    "prompt_tokens_details": null,
    "completion_tokens_details": {
      "text_tokens": null,
      "audio_tokens": null,
      "image_tokens": null,
      "reasoning_tokens": 0,
      "accepted_prediction_tokens": null,
      "rejected_prediction_tokens": null
    }
  },
  "object": "chat.completion",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "{\"name\": \"search_web\", \"arguments\": {\"query\": \"fun fact about the roman empire\"} }",
        "tool_calls": null,
        "function_call": null
      },
      "finish_reason": "stop"
    }
  ],
  "created": 1769527344,
  "system_fingerprint": null
}

[SPOOKEXE]

Got this issue too but mine is doing it in the thinking:

{
  "response": {
    "content": "",
    "thinking": "{\n  \"name\": \"todo_create\",\n  \"arguments\": {\n    \"items\": [\n      {\n        \"title\": \"Analyze project structure and dependencies\",\n        \"description\": \"Review the project configuration files (pyproject.toml, README.md) to understand the tech stack and dependencies\",\n        \"priority\": \"high\"\n      },\n      {\n        \"title\": \"Review existing agent modes and skills\",\n        \"description\": \"Examine the backend/agent/modes and backend/agent/skills directories to understand current capabilities\",\n        \"priority\": \"medium\"\n      },\n      {\n        \"title\": \"Create initial documentation or setup files\",\n        \"description\": \"Based on analysis, create necessary documentation or configuration files for the project\",\n        \"priority\": \"medium\"\n      }\n    ]\n  }\n}",
    "tool_calls": null,
    "model": "glm-4.7-flash:latest",
    "usage": null,
    "finish_reason": "stop"
  }

I used GLM-4-7-Flash-GGUF with llama.cpp and it works fine, don't know if its Ollama's side or litellm. I've also tested qwen3-coder-next, qwen3-vl and deepseek-r1 and they all have this issue. Even with "think": false being passed in, it still does it.

[SPOOKEXE]

And here glm-4.7-flash works fine with LM Studio

[DeansTagLab]

Hi,
I've the same issue.
When using gpt-oss:20b directly via ollama the tool calling is working and it search in my memory:

When going through Litellm everything is freezing and i got not response:

[rcmurphy]

I am no longer seeing this behavior in the latest version of OpenWebUI. @SPOOKEXE @DeansTagLab, please try upgrading and let me know whether the behavior persists or resolves. If I don't hear anything within the next couple of days, I'll close the issue.

EDIT: Configuration got messed up in OpenWebUI, disabling native tool calls. Still reproducible with native tool calling ☹️

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.