Docker Compose for Ollama + LiteLLM + Open WebUI

Start the ollama container first:

docker compose up -d ollama

Before you can use the models specified in the litellm config ./configs/litellm/config.yaml we need to pull the models first:

docker exec -it ollama sh -c 'ollama pull llama3.2:latest'
docker exec -it ollama sh -c 'ollama pull gemma:2b'

Then start the stack:

make up

Open WebUI can be access on port 8080 and you can setup your admin account on the frontend.

LiteLLM can be access on port 4000 and the password is defined under .env as LITELLM_MASTER_KEY="sk-example-master-key" by default.