Skip to content

Run Local Services in Docker

Jump to bottom
Jdaie edited this page Jan 29, 2026 · 4 revisions

You can use Docker to containerize and deploy ASR, LLM, and TTS services on your laptop or devices like the Pi5. Then, install the chatbot on a smaller device (such as the Pi Zero 2W) and connect these services over your local network.

This way, you can run whisplay-ai-chatbot locally without relying on external API services.

Containers

  • ollama
  • faster-whisper
  • piper-http

The ollama image is officially provided by Ollama, while the faster-whisper and piper-http images are built locally on your device using the provided Dockerfiles.

Setting Up Local Services with Docker

You should Install Docker on your device first. Follow the official Docker installation guide for your operating system: https://www.docker.com/

  1. git clone the whisplay-ai-chatbot repository:
git clone https://github.com/pisugar/whisplay-ai-chatbot.git
cd whisplay-ai-chatbot
  1. Navigate to the docker directory:
cd docker

# if you need to change model for faster-whisper or piper,
# edit the docker-compose.yml file accordingly before building the containers.

# if you have trouble accessing huggingface.io,
# you can use proxy settings in the docker-compose.yml file to set up a proxy for the model downloads.
docker compose build

# After building the containers, start the services:
docker compose up -d

Check if all the containers are running normally:

docker ps
  1. After the containers are up and running, pull the models for ollama, here I use the qwen3:1.7b model as an example:
docker exec -it ollama ollama pull qwen3:1.7b
  1. Now, you can configure the whisplay-ai-chatbot to connect to these local services. Edit the .env file:
# replace <docker-device-ip> with the IP address of the device running the Docker containers

ASR_SERVER=faster-whisper
LLM_SERVER=ollama
TTS_SERVER=piper-http

# ollama settings
OLLAMA_MODEL=qwen3:1.7b
OLLAMA_ENDPOINT=http://<docker-device-ip>:11434

# faster-whisper settings
FASTER_WHISPER_PORT=8803
FASTER_WHISPER_HOST=<docker-device-ip>
FASTER_WHISPER_REQUEST_TYPE=base64
FASTER_WHISPER_LANGUAGE=en
# no need to set the model path if you are not running faster-whisper on the same device as the chatbot
# FASTER_WHISPER_MODEL_SIZE_OR_PATH=/path/to/your/faster-whisper-model-directory

# piper-http settings
PIPER_HTTP_HOST=<docker-device-ip>
PIPER_HTTP_PORT=8805
PIPER_HTTP_MODEL=en_US-amy-medium
# specify the speech speed, default is 1. The larger the value, the slower the speech.
PIPER_HTTP_LENGTH_SCALE=1
  1. Finally, restart the whisplay-ai-chatbot:
sudo systemctl restart chatbot.service

Enjoy your local whisplay-ai-chatbot setup!

Changing whisper or piper models

If you want to change the models used in faster-whisper or piper-http, you need to edit the docker-compose.yml file in the docker directory and rebuild the containers.

Clone this wiki locally