Scriber

A self-hosted voice transcription service with optional summarization. Send audio via a web UI, REST API, or Telegram bot — everything runs locally on your own machine.

---

Features

• 🎤 Transcribe audio files or live microphone recordings via a web UI
• 🤖 Telegram bot integration — send voice messages, get transcripts back
• 🌐 REST API for programmatic access
• 📝 Optional summarization — local (no API key) or via Claude, OpenAI, Gemini
• 🔒 Fully self-hosted — audio never leaves your server (unless using a cloud summary provider)

---

Project Structure

whisper-app/
├── docker-compose.yml
├── .env
├── .gitignore
├── api/
│   ├── Dockerfile
│   ├── main.py
│   └── static/
│       └── index.html
└── bot/
    ├── Dockerfile
    └── bot.py

---

Prerequisites

• Docker and Docker Compose, or Podman with podman-compose
• A Telegram bot token (optional — only needed for the Telegram bot)

---

Setup

1. Clone or create the project

mkdir whisper-app && cd whisper-app
mkdir -p api/static bot cache/whisper cache/huggingface

Copy all project files into the appropriate directories as described in the Project Structure above.

2. Configure environment variables

Copy the example below into a .env file in the project root:

# Required for Telegram bot — omit if you don't need it
TELEGRAM_TOKEN=your-telegram-token-here

# Whisper model to use for transcription
# Options: tiny, base, small, medium, large
# Larger = more accurate but slower and more RAM
WHISPER_MODEL=base

# Optional — summarization provider
# Options: local, claude, openai, gemini
# Leave empty to disable summarization entirely
SUMMARY_PROVIDER=

# API keys — only the one matching SUMMARY_PROVIDER is needed
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GEMINI_API_KEY=

⚠️ Never commit .env to version control. It's already listed in .gitignore.

3. Get a Telegram bot token (optional)

If you want to use the Telegram bot:

1. Open Telegram and search for @BotFather
2. Send /newbot and follow the prompts
3. Copy the token you receive and paste it as TELEGRAM_TOKEN in .env

---

Deployment

Without summarization (transcript only)

docker compose up --build

With summarization

Set SUMMARY_PROVIDER in .env to one of the supported providers (see Summarization section below), then:

docker compose up --build

Run in the background

docker compose up --build -d

View logs:

docker compose logs -f

Stop everything:

docker compose down

---

Summarization

Summarization is fully optional. Set SUMMARY_PROVIDER in .env to enable it. If unset or empty, only the transcript is returned — no errors, no warnings.

Providers

Provider	Value	Privacy	Cost	Requires
Local (distilbart)	local	100% local	Free	~1.2 GB extra RAM
Claude (Anthropic)	claude	Anthropic servers	Pay per use	ANTHROPIC_API_KEY
ChatGPT (OpenAI)	openai	OpenAI servers	Pay per use	OPENAI_API_KEY
Gemini (Google)	gemini	Google servers	Free tier available	GEMINI_API_KEY

Local summarization

Uses sshleifer/distilbart-cnn-6-6 running in-process inside the whisper-api container — no extra service or API key needed. The model (~1.2 GB) is downloaded from Hugging Face on first use and cached locally in ./cache/huggingface.

SUMMARY_PROVIDER=local

Cloud summarization

Set the provider and the corresponding API key:

# Claude
SUMMARY_PROVIDER=claude
ANTHROPIC_API_KEY=sk-ant-your-key-here

# OpenAI
SUMMARY_PROVIDER=openai
OPENAI_API_KEY=sk-your-key-here

# Gemini
SUMMARY_PROVIDER=gemini
GEMINI_API_KEY=your-gemini-key-here

API keys can be obtained at:

• Claude → console.anthropic.com
• OpenAI → platform.openai.com
• Gemini → aistudio.google.com

---

Resource Requirements

Whisper models

Model	RAM needed	Speed	Accuracy
tiny	~1 GB	Very fast	Basic
base	~1 GB	Fast	Good
small	~2 GB	Moderate	Better
medium	~5 GB	Slow	Great
large	~10 GB	Very slow	Best

Local summarization model

Model	Size	Notes
sshleifer/distilbart-cnn-6-6	~1.2 GB	Downloaded automatically on first use

---

Model Caching

Models are cached on the host to avoid re-downloading on every run:

• Whisper models → ./cache/whisper
• Hugging Face models (local summarization) → ./cache/huggingface

Both folders are mounted into the container via docker-compose.yml. On first run each model downloads once; subsequent runs start instantly.

---

Platform Notes

macOS (Colima)

Docker on macOS runs inside a Linux VM with its own memory limit. If you're running large models, increase the Colima VM resources before starting:

colima stop
colima start --memory 12 --cpu 4

Fedora / Podman

Use podman compose instead of docker compose throughout. Podman runs rootless by default, so volume mounts require the :Z SELinux label and correct ownership:

mkdir -p ./cache/whisper ./cache/huggingface
podman unshare chown -R 0:0 ./cache/whisper ./cache/huggingface

Also ensure aardvark-dns is installed for container name resolution:

sudo dnf install aardvark-dns

Debian / Linux (Docker)

If containers can't reach the internet (DNS resolution failures), enable IP forwarding:

echo "net.ipv4.ip_forward=1" > /etc/sysctl.d/99-docker.conf
sysctl -p /etc/sysctl.d/99-docker.conf
systemctl restart docker

---

Usage

Web UI

Open your browser at:

http://localhost:8000

Upload an audio file or record directly from your microphone. The transcript appears immediately, with the summary shown above it when a provider is configured.

REST API

Transcribe an audio file:

curl -X POST http://localhost:8000/transcribe \
  -F "file=@your_audio.mp3"

Example response (with summarization enabled):

{
  "transcript": "Hey, just a reminder about the meeting tomorrow at 3pm.",
  "summary": "A reminder about a meeting scheduled for tomorrow at 3pm.",
  "language": "en",
  "duration": 4.2
}

When summarization is disabled, summary is null.

Health check:

curl http://localhost:8000/health

{
  "status": "ok",
  "whisper_model": "base",
  "summarization": "local"
}

Telegram Bot

1. Open Telegram and find your bot by its username
2. Send /start to confirm it's running
3. Send any voice message — you'll receive the transcript and (optionally) a summary in reply

---

Configuration Reference

Variable	Required	Default	Description
TELEGRAM_TOKEN	No	—	Telegram bot token from @BotFather
WHISPER_MODEL	No	base	Whisper model: tiny, base, small, medium, large
SUMMARY_PROVIDER	No	—	Summary backend: local, claude, openai, gemini
ANTHROPIC_API_KEY	No	—	Required when SUMMARY_PROVIDER=claude
OPENAI_API_KEY	No	—	Required when SUMMARY_PROVIDER=openai
GEMINI_API_KEY	No	—	Required when SUMMARY_PROVIDER=gemini

---

Troubleshooting

Container runs out of memory

Reduce the Whisper model size in .env (small instead of large), or increase Docker/Colima/Podman memory limits.

http://localhost:8000 is unreachable

Check that the container is running and uvicorn bound correctly:

docker compose ps
docker compose logs whisper-api

Summary is null but provider is set

Check the whisper-api logs for warnings:

docker compose logs whisper-api

Common causes: API key missing or wrong, network issue reaching the cloud provider, or local model still downloading on first run.

Whisper or Hugging Face model re-downloads on every run

Make sure the cache volumes are correctly mounted and the folders exist on the host:

ls ./cache/whisper
ls ./cache/huggingface

If empty, check volume mount permissions (Podman users: run podman unshare chown -R 0:0 ./cache/whisper ./cache/huggingface).

Telegram bot doesn't respond

Verify the token is correct and the bot container is running:

docker compose logs telegram-bot

---

License

MIT