Flow Provider with Ollama: /api/generate returns empty AI Chat response, /api/chat fails Test Connection with 404

[ranjit3770]

Environment

• Flow Provider: Ollama
• Ollama Model: gpt-oss:20b

Issue

I'm experiencing two different issues when configuring the Ollama Flow Provider.

Case 1: Using /api/generate

Endpoint

http://[OLLAMA-SERVER]/api/generate

Behavior

• ✅ Test Connection succeeds.
• ❌ AI Chat returns an empty response (no output).

---

Case 2: Using /api/chat

Endpoint

http://[OLLAMA-SERVER]/api/chat

Behavior

• ❌ Test Connection fails with:

litellm.APIConnectionError: OllamaException - 404 page not found

---

Verification

The same /api/chat endpoint works correctly when tested directly using Postman/cURL.

curl --location 'http://[OLLAMA-SERVER]/api/chat' \
--header 'Content-Type: application/json' \
--data '{
  "model": "gpt-oss:20b",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "stream": false
}'

Response

{
  "model": "gpt-oss:20b",
  "created_at": "2026-07-22T08:17:24.694493298Z",
  "message": {
    "role": "assistant",
    "content": "Hello! 👋 How can I help you today?",
    "thinking": "The user says \"Hello!\". We should respond with a friendly greeting."
  },
  "done": true,
  "done_reason": "stop",
  "total_duration": 837122888,
  "load_duration": 282868911,
  "prompt_eval_count": 83,
  "prompt_eval_duration": 40953169,
  "eval_count": 59,
  "eval_duration": 450009439
}

Expected Behavior

• When using http://[OLLAMA-SERVER]/api/chat, the Test Connection should succeed.
• AI Chat should return the assistant response correctly.
• If /api/generate is supported, AI Chat should also produce the generated response instead of returning an empty message.

Actual Behavior

Endpoint	Test Connection	AI Chat
/api/generate	✅ Success	❌ Empty response
/api/chat	❌ litellm.APIConnectionError: OllamaException - 404 page not found	Cannot test

Additional Notes

• The Ollama server is reachable.
• The model is loaded and responds correctly.
• Direct requests to the Ollama API using Postman/cURL work as expected.
• The issue appears to be related to how the Flow Provider (or LiteLLM integration) is constructing or validating requests for Ollama endpoints, rather than an issue with the Ollama server itself.

[ShrihariMahabal]

Can you try model_id: ollama_chat/gpt-oss:20b
API base URL: http://[OLLAMA-SERVER]:11434 (no /api/...)

[ranjit3770]

Can you try model_id: ollama_chat/gpt-oss:20b API base URL: http://[OLLAMA-SERVER]:11434 (no /api/...)

Hi, thanks for the suggestion.

I tested it with the following configuration:

• Model ID: ollama_chat/gpt-oss:20b
• API Base URL: http://[OLLAMA-SERVER]:11434 (without /api/...)

Here are the results:

1. API Key

   • The UI indicates that the API key can be left empty for a local Ollama server.

   • However, when left empty, the test connection fails with:

     litellm.APIConnectionError: OllamaException - Illegal header value b'Bearer '
     

   • As a workaround, I entered a dummy API key, after which the connection test succeeds.

2. Model ID

   • When I specify ollama_chat/gpt-oss:20b, the system automatically prepends the provider name, resulting in:

     ollama/ollama_chat/gpt-oss:20b
     

   • This causes a model not found error because the resulting model name is incorrect.

3. Without /api/... in the endpoint

   • Using only http://[OLLAMA-SERVER]:11434, the connection test succeeds.
   • However, AI Chat still returns an empty response, which is the original issue.

So there appear to be two separate issues:

• An empty API key results in an invalid Authorization: Bearer header.
• Even with a successful connection (using a dummy API key and the base URL), AI Chat still produces an empty response.