/ollama/v1/chat/completions endpoint broke (405 Method Not Allowed) in v0.3.6

[kobaltz]

Bug Report

Description

Bug Summary:
Making a HTTP POST to /ollama/v1/chat/completions worked in v0.3.5 but now returns 405 - Method not allowed

Steps to Reproduce:
Using the docker compose file which at the time was referencing v0.3.6, I was unable to make API calls to the Chat Completion
Reverting the open webui image to v0.3.5 and the POST works as expected.

services:
  ollama:
    volumes:
      - ollama:/root/.ollama
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:${OLLAMA_DOCKER_TAG-latest}
    environment:
      - NVIDIA_VISIBLE_DEVICES=all
      - NVIDIA_DRIVER_CAPABILITIES=all
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
  open-webui:
    build:
      context: .
      args:
        OLLAMA_BASE_URL: '/ollama'
      dockerfile: Dockerfile
    image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG-main}
    container_name: open-webui
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - ${OPEN_WEBUI_PORT-2039}:8080
    environment:
      - 'OLLAMA_BASE_URL=http://ollama:11434'
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}
  open-webui: {}

Expected Behavior:
HTTP POST to /ollama/v1/chat/completions returns the JSON response

Actual Behavior:
405 Method Not Allowed when making HTTP POST to /ollama/v1/chat/completions

Environment

• Open WebUI Version: main (and v0.3.6)

• Ollama (if applicable): latest (and 0.1.47)

• Operating System: Debian Bookworm

• Browser (if applicable): n/a

Reproduction Details

Confirmation:

• I have read and followed all the instructions provided in the README.md.
• I am on the latest version of both Open WebUI and Ollama.
• I have included the Docker container logs.

Logs and Screenshots

Docker Container Logs:

INFO:apps.ollama.main:url: http://ollama:11434
INFO:     my-ip:0 - "POST /ollama/v1/chat/completions HTTP/1.1" 405 Method Not Allowed
INFO:apps.ollama.main:url: http://ollama:11434
INFO:     my-ip:0 - "POST /ollama/v1/chat/completions HTTP/1.1" 405 Method Not Allowed

Installation Method

Docker Compose

[debemdeboas]

Seeing this here as well.
Also seeing a TypeError: signup() missing 1 required positional argument: 'form_data'.
Going back to v0.3.5 fixes things.

[justinh-rahb]

Can you share the exact cURL command you've used to test, as well as additional browser dev console logs, maybe some more backend logs as well to help us out.

[kobaltz]

Here's the cURL command.

curl -X POST \
  http://127.0.0.1:2039/ollama/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-89ddf4d66bae4527974150ee796e71f6' \
  -d '{
       "model": "llama3:instruct",
       "messages": [
         {
           "role": "user",
           "content": "Tell me a story"
         }
       ],
       "max_tokens": 50
     }'
     
=> {"detail":"Method Not Allowed"}

But, if I change the docker compose file to v0.3.5 and run the same command, I get

=> {"id":"chatcmpl-138","object":"chat.completion","created":1719588557,"model":"llama3:instruct","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":"Once upon a time, in a small village nestled in the rolling hills of Tuscany, there was a little girl named Sophia. She lived with her family in a charming stone house with a bright red door and a garden full of fragrant herbs"},"finish_reason":"length"}],"usage":{"prompt_tokens":0,"completion_tokens":50,"total_tokens":50}}

Since I'm using just the cURL command, there isn't any additional logs as the docker container logas are just saying that the method isn't allowed. Maybe there is a debug ENV var I could add to get a more verbose output?

[abishekmuthian]

I'm getting Method Not Allowed for deepseek-coder-v2:latest model in v0.3.6. Other models work fine.

INFO: 127.0.0.1:41212 - "POST /ollama/api/chat HTTP/1.1" 405 Method Not Allowed

[oleksii-davydchuk]

I'm getting Method Not Allowed for any models in v0.3.6
I used local k8s cluster, which was created with kind I got Server connection verified for the Ollama API url ollama.ollama.svc.cluster.local:11434

INFO:apps.ollama.main:app.state.config.OLLAMA_BASE_URLS: ['http://ollama.ollama.svc.cluster.local:11434']
INFO:     127.0.0.1:56508 - "POST /ollama/urls/update HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:apps.openai.main:get_all_models()
INFO:     127.0.0.1:56508 - "POST /openai/keys/update HTTP/1.1" 200 OK
INFO:     127.0.0.1:56494 - "GET /ollama/api/version HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:     127.0.0.1:56508 - "GET /api/models HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:     127.0.0.1:56494 - "GET /api/models HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:     127.0.0.1:56546 - "GET /ollama/api/version HTTP/1.1" 200 OK
INFO:     127.0.0.1:56552 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:     127.0.0.1:56562 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:url: http://ollama.ollama.svc.cluster.local:11434
INFO:     127.0.0.1:52376 - "POST /ollama/api/pull/0 HTTP/1.1" 200 OK
INFO:apps.ollama.main:get_all_models()
INFO:apps.ollama.main:get_all_models()
INFO:     127.0.0.1:52376 - "GET /api/models HTTP/1.1" 200 OK
INFO:     127.0.0.1:57818 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO:     127.0.0.1:57818 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO:apps.ollama.main:url: http://ollama.ollama.svc.cluster.local:11434
INFO:     127.0.0.1:57818 - "POST /ollama/api/chat HTTP/1.1" 405 Method Not Allowed
ERROR:asyncio:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0xfffe9473a910>
INFO:     127.0.0.1:57818 - "GET /api/v1/chats/ HTTP/1.1" 200 OK

[oleksii-davydchuk]

Issue was resolved - there was a memory shortage and the OOM kicked in.

[tjbck]

Fixed on dev, 0.3.7 coming soon!

[Lanhild]

Commenting about this, I am getting an error 500 with the completions endpoint:

Updating ollama to 0.3.9 fixed this.