A FastAPI server that exposes WhisperX as an OpenAI-compatible audio transcription API. Supports both a simple single-server mode and a horizontally scalable distributed mode backed by Kafka and S3.
- OpenAI-compatible — drop-in replacement for
/v1/audio/transcriptionsand/v1/audio/translations - Alignment & diarization — word-level timestamps and speaker labels out of the box
- Multiple output formats —
json,verbose_json,vtt_json,srt,vtt,aud,text - Distributed mode — offload GPU work to dedicated workers via Kafka + S3 (MinIO)
- Live request status — poll an in-flight transcription's current pipeline stage by request id, in both direct and Kafka modes
- Pluggable backends — swap transcription, alignment, and diarization implementations per stage
- API key auth — single key or a JSON key-map for multi-client setups
- Optional web UI — a built-in browser client (upload, live stage progress, interactive transcript, exports), off by default
Each profile selects exactly one combination — service set, image tags, and --extra build args are all driven by the profile so images stay minimal.
# 1. Normal CUDA → image: whisperx-api:cuda
docker compose --profile cuda up
# 2. Normal CPU → image: whisperx-api:cpu
docker compose --profile cpu up
# 5. CUDA + Observability → image: whisperx-api:cuda-metrics (+ Prometheus)
docker compose --profile cuda-observe up
# 6. CPU + Observability → image: whisperx-api:cpu-metrics (+ Prometheus)
docker compose --profile cpu-observe upThe API is available at http://localhost:8000. With an *-observe profile, Prometheus is on :9090 (point your own Grafana / dashboard at it).
# Copy and edit credentials before first run
cp .env.example .env
# 3. CUDA + Kafka → api: whisperx-api:kafka, worker: whisperx-worker:cuda-kafka
docker compose -f compose-kafka.yaml --profile cuda up
# 4. CPU + Kafka → api: whisperx-api:kafka, worker: whisperx-worker:cpu-kafka
docker compose -f compose-kafka.yaml --profile cpu up
# 7. CUDA + Kafka + Observability → api: whisperx-api:kafka-metrics, worker: whisperx-worker:cuda-kafka-metrics
docker compose -f compose-kafka.yaml --profile cuda-observe up
# 8. CPU + Kafka + Observability → api: whisperx-api:kafka-metrics, worker: whisperx-worker:cpu-kafka-metrics
docker compose -f compose-kafka.yaml --profile cpu-observe upWorkers process one job at a time per container. Scale horizontally by running multiple worker replicas. On
SIGTERM(stop / rolling update) a worker finishes its in-flight job — reply, then offset commit — before exiting, and refuses to start a new one. Set the containerstop_grace_periodto your worst-case job duration so a long job isn't killed mid-flight; a job killed before commit is safely redelivered (idempotent resend) at the cost of one reprocess.
Delivery is at-least-once. A worker writes each job's result envelope to S3 (
results/{job_id}) before replying, so a redelivered job resends the stored reply instead of re-running. Each job is guarded by a processing lease (claims/{job_id}, TTLKAFKA__JOB_LEASE_TTL_SECONDS, default 300s): a copy redelivered mid-run (e.g. after a consumer-group rebalance) defers instead of starting a concurrent duplicate, and takes over only once the lease expires. The lease also counts delivery attempts — a job that repeatedly kills its worker is, pastKAFKA__MAX_DELIVERY_ATTEMPTS(default 3), routed to thetranscription-dlqtopic so the submitter fails fast instead of every worker dying on it. Bothresults/andclaims/objects expire via the bucket lifecycle (S3__OBJECT_EXPIRY_DAYS, default 1 day).
| # | Mode | Profile | Compose file |
|---|---|---|---|
| 1 | Normal CUDA | cuda |
compose.yaml |
| 2 | Normal CPU | cpu |
compose.yaml |
| 3 | CUDA + Kafka | cuda |
compose-kafka.yaml |
| 4 | CPU + Kafka | cpu |
compose-kafka.yaml |
| 5 | CUDA + Observability | cuda-observe |
compose.yaml |
| 6 | CPU + Observability | cpu-observe |
compose.yaml |
| 7 | CUDA + Kafka + Observability | cuda-observe |
compose-kafka.yaml |
| 8 | CPU + Kafka + Observability | cpu-observe |
compose-kafka.yaml |
A single-page browser client for the API lives in webui/ (React + Vite + TypeScript + Tailwind). It is a thin client over the public endpoints — it adds no API routes and no Python dependencies, and the core API behaves identically whether it is enabled or not (covered by a test). Features: drag-and-drop upload with progress, all transcription form parameters, a live pipeline-stage timeline (client-generated X-Request-ID + status polling, working in both direct and Kafka modes), an interactive transcript with click-to-seek, word-level highlighting and color-coded speakers, export buttons for every response format, one-click re-run of the same file with adjusted parameters (the file stays loaded in the browser — no re-selection), links to the interactive API docs (Swagger UI / ReDoc), and a light/dark theme toggle (follows the OS by default).
Enable it with WEBUI__ENABLED=true. The UI is then served at /webui/ (with / redirecting to it); when the flag is off — the default — neither route exists. The server fails fast at startup if the flag is on but no build output is found.
Docker images build the UI via an opt-in build arg (default skip — no bun stage runs, existing profiles build exactly as before):
# Standalone; same BUILD_WEBUI=build works for compose-kafka.yaml
BUILD_WEBUI=build docker compose --profile cuda build
# then run with WEBUI__ENABLED=true in .env
docker compose --profile cuda upWithout Docker, build the assets once (Bun ≥ 1.3; no Bun process is needed at runtime):
cd webui && bun install --frozen-lockfile && bun run build
WEBUI__ENABLED=true whisperx-apiThe server looks for webui/dist relative to the working directory, then the repo root; point WEBUI__DIST_DIR at the build output for non-standard layouts. For UI development, bun run dev starts Vite on :5173 and proxies API calls to http://localhost:8000 (override with WEBUI_DEV_API). The frontend carries its own quality gates — bun run lint (Biome) and bun run test (Vitest) — which run in CI on every push.
Auth: the static assets themselves are served without an API key (they contain nothing sensitive); every API call the UI makes carries the key the user enters. The UI probes GET /info on load — a 200 without credentials means no key is configured and the key field is hidden.
All settings are environment variables. Nested fields use __ as a delimiter (e.g. WHISPER__MODEL=large-v3).
All available settings are defined in config.py. Variables you'll most likely need to set:
| Variable | Default | Description |
|---|---|---|
WHISPER__MODEL |
large-v3 |
Transcription model name |
WHISPER__COMPUTE_TYPE |
default |
Quantization — float16 for GPU, float32 for CPU |
WHISPER__INFERENCE_DEVICE |
auto |
cpu, cuda, or auto |
HF_TOKEN |
— | Hugging Face token (required for pyannote diarization) |
API_KEY |
— | Single API key for all requests |
API_KEYS_FILE |
— | Path to JSON file mapping key → client name |
AUTH_REQUIRED |
false |
When true, refuse to start unless API_KEY or API_KEYS_FILE is set |
MODE |
direct |
direct or kafka |
MAX_CONCURRENT_TRANSCRIPTIONS |
1 |
Max concurrent ML inferences (transcribe / align / diarize). 0 = unlimited. See the concurrency note below. |
WEBUI__ENABLED |
false |
Serve the bundled web UI at /webui |
MAX_CONCURRENT_TRANSCRIPTIONSparallelizes across distinct models, not within one. Each transcription pipeline holds a per-model lock during the transcribe step, so two requests for the same model still run one at a time even with the limit raised — the setting lets a request for a different model (and the align / diarize stages) proceed concurrently. To raise same-model throughput, add GPUs or run more replicas / workers. Whichever process runs inference (the API in direct mode, the worker in Kafka mode) logs this caveat at startup when the limit is >1.
Auth is off by default. With neither
API_KEYnorAPI_KEYS_FILEset, the server accepts every request without credentials and logs a startup warning. Set either to enforce auth (missing header → 401, invalid key → 403), or setAUTH_REQUIRED=trueto turn an unconfigured deployment into a startup failure rather than an open server.
Additional variables for Kafka mode:
| Variable | Default | Description |
|---|---|---|
KAFKA__BOOTSTRAP_SERVERS |
localhost:9092 |
Kafka broker address |
KAFKA__PROGRESS_TOPIC |
transcription-progress |
Best-effort topic for per-stage worker progress events consumed by the status endpoint |
S3__ENDPOINT_URL |
http://localhost:9000 |
S3 / MinIO endpoint |
S3__BUCKET |
whisperx-audio |
Bucket for audio uploads |
MINIO_ROOT_USER |
minioadmin |
MinIO root user — change before deploying |
MINIO_ROOT_PASSWORD |
minioadmin |
MinIO root password — change before deploying |
Status-endpoint tuning (both modes):
| Variable | Default | Description |
|---|---|---|
REQUEST_STATUS__TTL_SECONDS |
300 |
How long terminal states (completed / failed) are retained for polling |
REQUEST_STATUS__MAX_ENTRIES |
4096 |
Hard cap on tracked requests; terminal entries are evicted first when over capacity |
REQUEST_STATUS__CLEANUP_INTERVAL_SECONDS |
30 |
How often the background sweep evicts expired entries |
Transcribe an audio file. Compatible with the OpenAI transcription API.
Form parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
file |
file | — | Audio file (required) |
model |
string | config default | Model name. whisper-1 is aliased to the configured default. |
language |
string | config default | ISO-639-1 language code. Auto-detected if omitted. |
prompt |
string | — | Optional context/hotwords hint |
response_format |
string | json |
text, json, verbose_json, vtt_json, srt, vtt, aud |
temperature |
float | 0.0 |
Sampling temperature |
timestamp_granularities[] |
list | ["segment"] |
segment, word |
align |
bool | true |
Enable word-level alignment (required for subtitle formats) |
diarize |
bool | false |
Enable speaker diarization (requires align=true) |
speaker_embeddings |
bool | false |
Include speaker embeddings in diarization output |
min_speakers |
int | — | Lower bound on the speaker count for diarization (≥ 1) |
max_speakers |
int | — | Upper bound on the speaker count for diarization (≥ min_speakers) |
highlight_words |
bool | false |
Highlight words in vtt/srt output |
suppress_numerals |
bool | true |
Spell out numbers |
hotwords |
string | — | Comma-separated hotwords to bias toward |
batch_size |
int | config default | Inference batch size |
chunk_size |
int | config default | VAD chunk size in seconds |
async |
bool | false |
Kafka mode only. Return 202 Accepted immediately instead of blocking; fetch the outcome later. See Async job submission. |
callback_url |
string | — | Optional URL the result envelope is POSTed to when the job finishes. Validated against the same SSRF policy as audio_url. See Completion webhook. |
Response formats
| Format | Content-Type | Body |
|---|---|---|
json |
application/json |
{"text": "..."} |
verbose_json |
application/json |
Full transcript with segments and timestamps |
vtt_json |
application/json |
verbose_json + "vtt_text" field |
text / srt / aud |
text/plain |
Raw text / subtitle file |
vtt |
text/vtt |
WebVTT subtitle file |
Translate audio to English. Same parameters as /v1/audio/transcriptions, minus language, align, diarize, and diarization-related fields.
Return the live processing stage for a transcription request. Useful for surfacing a "still working — currently transcribing" indicator in long-running UIs.
Because the transcription POST is synchronous (the response only arrives when the whole pipeline finishes), the client must set its own id on the POST so it can poll status in parallel:
# Submit with a known id
curl -X POST http://localhost:8000/v1/audio/transcriptions \
-H 'X-Request-ID: my-request-1' \
-F file=@audio.mp3 -F model=large-v3 -F align=true &
# Poll status from another shell
curl http://localhost:8000/v1/audio/transcriptions/my-request-1/statusThe middleware accepts client-supplied X-Request-ID values matching [A-Za-z0-9._-]{1,128}; anything else is rejected and replaced with a server-generated UUID (returned via the response header, by which point it is too late to poll).
Response (200)
Stages
| Mode | Stage names |
|---|---|
| direct | upload_save, audio_load, awaiting_concurrency, transcribe, align, diarize, finalize |
| kafka (API-side) | uploading_to_s3, submitted_to_kafka, awaiting_worker |
kafka (worker-side, via transcription-progress topic) |
worker.s3_download, worker.audio_load, worker.awaiting_gpu, worker.transcribe, worker.align, worker.diarize, worker.finalize |
Failures (invalid audio, queue full, timeout, worker error, …) end the lifecycle with status="failed" and populate error / error_type. Stages completed before the failure are preserved.
Terminal states (completed / failed) are retained for REQUEST_STATUS__TTL_SECONDS (default 300s) so polling clients that arrive just after the POST returns can still confirm the outcome. After that, the id 404s.
Other responses
400 Bad Request— malformedrequest_id(must match[A-Za-z0-9._-]{1,128})404 Not Found— id is unknown, expired, or not yet seen by this replica
Both the request/reply path and
/statusscale across API replicas in Kafka mode. Each reply is delivered to every replica and only the one holding the job resolves it, so any replica can serve the POST. Status converges too: the submitting replica announces the job on the progress topic and every replica consumes the progress stream, soGET /statusworks on any replica behind a load balancer — no sticky sessions required. Each replica's tracker therefore holds entries for jobs across all replicas (bounded byREQUEST_STATUS__MAX_ENTRIES, default 4096, well aboveKAFKA__MAX_PENDING_JOBS). In-flight status is still in-memory: a replica restart loses the live history for jobs it learned about (they would expire within minutes anyway), while completed/failed outcomes stay readable forREQUEST_STATUS__TTL_SECONDS. Direct mode is single-process and unaffected.
Streams the processing status of a transcription request as Server-Sent Events. Each state change is emitted as a data: line carrying the status JSON — status (queued / in_progress / completed / failed), the active stage, the per-stage stages[] timeline, and error / error_type on failure. : ping comment lines are sent while idle to hold the connection open. The stream ends when the request reaches a terminal state, the client disconnects, or REQUEST_STATUS__SSE_MAX_DURATION_SECONDS elapses.
The client sets a known X-Request-ID (matching [A-Za-z0-9._-]{1,128}) on the transcription POST and opens the stream alongside it:
curl -N http://localhost:8000/v1/audio/transcriptions/my-request-1/eventsWhen auth is configured the endpoint requires the API key like every other route, so consume it with a header-capable client rather than a browser EventSource. A malformed id returns 400 Bad Request; an unknown, expired, or not-yet-tracked id returns 404 Not Found.
By default the transcription POST is synchronous — the HTTP response arrives only when the whole pipeline finishes. In Kafka mode you can instead submit the job and return immediately by setting the async form field, then fetch the result later. This decouples slow transcriptions from the request connection (no client read-timeout to tune) and survives an API-replica restart, because the result is read from durable storage rather than an in-memory future.
# Submit — returns 202 without waiting for transcription
curl -X POST http://localhost:8000/v1/audio/transcriptions \
-H 'X-Request-ID: my-async-1' \
-F file=@audio.mp3 -F model=large-v3 -F align=true \
-F async=trueResponse (202 Accepted)
{
"request_id": "my-async-1",
"status": "accepted",
"status_url": "/v1/audio/transcriptions/my-async-1/status",
"result_url": "/v1/audio/transcriptions/my-async-1/result"
}Poll status_url (see above) to follow progress, then fetch result_url once status is completed. Unlike the synchronous path, you don't need to set X-Request-ID up front — the 202 body returns the resolved request_id and both URLs immediately, whether the id was client-supplied or server-generated (a uuid4). Supplying your own X-Request-ID (matching [A-Za-z0-9._-]{1,128}) is optional: a predictable id for log correlation. Async is rejected in direct mode with 400 Bad Request.
Fetch and (re-)format the final result of a finished transcription without re-running the pipeline. In Kafka mode it reads the stored results/{job_id} envelope from S3, so it works from any replica and after restarts. In direct mode it reads an on-disk result store written when the synchronous POST completes — enabled by default (RESULT_STORE__ENABLED), bounded by count and age, and best-effort (a temp dir, not shared across replicas or guaranteed across restarts).
Query parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
response_format |
string | config default | Same formats as the POST: text, json, verbose_json, vtt_json, srt, vtt, aud |
highlight_words |
bool | false |
Highlight words in vtt/srt output |
Formatting is applied at fetch time from these query parameters — the response_format set on the async POST is not stored, so you choose the format (and may request several) when you fetch.
Responses
200 OK— formatted transcription (Content-Type perresponse_format), identical to what the synchronous POST would have returned400 Bad Request— malformedrequest_id404 Not Found— result not available (still pending, unknown, expired, or — in direct mode — never stored / result store disabled)- worker failures are replayed with the same status code the synchronous endpoint returns (e.g. invalid audio →
422, timeout →504), not404
Kafka-mode results are retained for S3__OBJECT_EXPIRY_DAYS (default 1 day) via the bucket lifecycle policy; direct-mode results for RESULT_STORE__TTL_SECONDS (default 1h), up to RESULT_STORE__MAX_ENTRIES files. After that the id 404s.
Add a callback_url form field to any transcription POST to have the result pushed to you when the job finishes, instead of (or alongside) polling. The URL is validated up front against the same SSRF policy as audio_url (URL_FETCH_ALLOW_PRIVATE_HOSTS / URL_FETCH_ALLOWED_HOSTS); a rejected host fails the request with 422 before any work starts.
On completion the server sends a single POST to callback_url with the terminal envelope as the JSON body:
{
"job_id": "my-async-1",
"status": "ok", // or "error"
"result": { }, // raw transcript (segments, language); absent on error
"error": "...", // present only when status is "error"
"error_type": "..."
}Format the result yourself, or ignore the body and fetch result_url for a formatted response.
- Who delivers: in Kafka mode the worker delivers — it is the single point that runs the job and holds the envelope, and it survives an API-replica restart. In direct mode the API delivers in-process after returning the synchronous response.
- Both outcomes notify (Kafka mode): success, handled failures, and jobs retired to the dead-letter queue all fire the webhook with the matching envelope. Direct mode fires on success only — a direct-mode failure is already returned to the (synchronously waiting) caller as an HTTP error.
- Delivery guarantee: best-effort with one retry, bounded by
WEBHOOK_TIMEOUT_SECONDS(default 15s). It fires at most once per fresh completion and is never re-sent on the redelivery/marker-resend path, so a worker restart between the reply and the callback can drop it. Treat the durableresult_urlas the source of truth and de-duplicate onjob_id. A non-2xx response or connection error is logged and dropped, not retried indefinitely.
Returns the running version, mode, uptime, concurrency / queue state, and (in Kafka mode) discovered worker membership. Also reports max_upload_size_bytes (null = unlimited) and subtitle_formats_available (whether this process can render vtt/srt/aud/vtt_json), so a client can pre-validate uploads and format support. Add ?detail=full for extended Kafka topology.
Returns {"status": "healthy"}. Not protected by API key auth.
| Endpoint | Description |
|---|---|
GET /models/catalog |
List known transcription model names + configured default (both modes) |
GET /models/list |
List loaded transcription models |
POST /models/load |
Load a model (model param) |
POST /models/unload |
Unload a model (model param) |
GET /align_models/list |
List loaded alignment models |
POST /align_models/load |
Load an alignment model (language param) |
POST /align_models/unload |
Unload an alignment model (language param) |
GET /diarize_models/list |
List loaded diarization models |
POST /diarize_models/load |
Load a diarization model (model param) |
POST /diarize_models/unload |
Unload a diarization model (model param) |
Each pipeline stage (transcription, alignment, diarization) can use a different backend. Set the active backend via environment variables:
BACKENDS__TRANSCRIPTION=whisperx
BACKENDS__ALIGNMENT=whisperx
BACKENDS__DIARIZATION=whisperxOnly the whisperx backend ships by default. Custom backends can be registered via the backend registry at src/whisperx_api_server/backends/.
| File | Purpose |
|---|---|
compose.yaml |
Standalone server — profiles: cuda, cpu, cuda-observe, cpu-observe |
compose-kafka.yaml |
Distributed stack (API + Kafka + MinIO + workers) — same four profiles |
Every runtime variant is gated by exactly one profile, so docker compose up never accidentally starts a GPU process on a machine that doesn't have one and observability stacks never spawn duplicate API servers.
# API server for the distributed (Kafka) setup — no ML dependencies required:
pip install ".[kafka]"
# API server with local inference (direct mode):
pip install ".[cpu]" # or ".[cuda]" for GPU
# Kafka worker:
pip install ".[cpu,kafka]" # or ".[cuda,kafka]"Two console scripts are installed:
whisperx-api # start the API server (UVICORN_HOST:UVICORN_PORT, default 0.0.0.0:8000)
whisperx-worker # start a Kafka workerWithout the cpu/cuda extras, PyTorch and WhisperX are not installed. Such a server can still take requests and hand them to workers in Kafka mode, but direct-mode inference and the subtitle response formats (srt, vtt, vtt_json, aud) need the ML extras and return a clear error otherwise.
Issues, forks, and pull requests are welcome.
GNU General Public License v3.0 — see LICENSE for details.
{ "request_id": "my-request-1", "status": "in_progress", // queued | in_progress | completed | failed "mode": "direct", // or "kafka" "stage": "transcribe", // name of the active stage "submitted_at": 1779198758.12, "updated_at": 1779198764.16, "completed_at": null, // set when status is terminal "filename": "audio.mp3", "stages": [ {"name": "upload_save", "duration_seconds": 0.003, "started_at": 1779198758.12, "completed_at": 1779198758.12, "in_progress": false}, {"name": "audio_load", "duration_seconds": 0.374, "started_at": 1779198758.12, "completed_at": 1779198758.49, "in_progress": false}, {"name": "awaiting_concurrency", "duration_seconds": 0.0, "started_at": 1779198758.49, "completed_at": 1779198758.49, "in_progress": false}, {"name": "transcribe", "started_at": 1779198758.49, "in_progress": true} ], "error": null, "error_type": null }