transcribe

Speaker-labelled transcripts from any audio or video file.

$ transcribe -d interview.mp4 -
[00:00:12] speaker 2: Oh, yeah. So a very common thing in enterprise software is...
[00:00:23] speaker 1: you spend a lot of time and energy using this product, but then...

Each tool does what it is good at:

1. ffmpeg converts the input to 16 kHz mono s16 wav
2. whisper.cpp transcribes it into timestamped segments (-oj json)
3. pyannote.audio diarizes the same wav into speaker turns
4. each segment takes the speaker it overlaps the most, and consecutive same-speaker segments merge into one printed turn

Without -d none of steps 3–4 happen, whisper-cli's own output is passed through untouched, and nothing from PyPI is needed at all.

Usage

transcribe [-d] [-s N] [--min-speakers N] [--max-speakers N] [-m MODEL]
           <input> [output] [-- whisper-cli args...]

The input is anything ffmpeg can decode — wav, mp3, m4a, flac, ogg, opus, aiff, mp4, mov, mkv — audio or video, any sample rate, mono or stereo. It is converted to 16 kHz mono s16 in a temp dir; the source file is never touched. A file that is already a 16 kHz mono s16 wav skips the conversion.

Output defaults to <input>.txt; - means stdout. Anything after a bare -- goes to whisper-cli verbatim (transcribe talk.m4a -- -l auto -ml 120).

flag	meaning
-d, --diarize	label each turn with a speaker
-s N, --speakers N	exact speaker count (default: let pyannote decide)
--min-speakers N / --max-speakers N	bound the speaker count instead of fixing it
-m, --model	ggml model name (large-v3-turbo) or path to a .bin
--pipeline	pyannote pipeline to run
--device	auto (default), cpu, mps, cuda
--no-download	never fetch a missing model
--whisper-cli	path to the whisper-cli binary
--enroll NAME	learn a voice from <input> and exit (see below)
--from-speaker N	which diarized speaker to enroll
--voices PATH	enrolled voice store
--threshold F	similarity needed to name a speaker (0.55)
--no-identify	number the speakers even when their voices are enrolled

Env equivalents: WHISPER_CLI, WHISPER_MODEL, PYANNOTE_PIPELINE, PYANNOTE_DEVICE, TRANSCRIBE_VOICES.

A bare model name is cached in ~/.cache/whisper.cpp/ and fetched from HuggingFace on first use, the same place download-ggml-model.sh gets it. The default is base.en, which is weak — -m large-v3-turbo is worth the download for anything you intend to read.

Naming the speakers

Diarization says these turns are the same person, not who. Enroll a voice once and it gets its name in every later transcript:

transcribe --enroll will --from-speaker 2 "some recording.m4a"
transcribe -d -m large-v3-turbo "another recording.m4a"

[00:02:01] zixy: You mentioned something about compliant and your experience with Hydra earlier.
[00:02:06] will: Well, yeah. I mean, I started going on a rant about, you know, the reason why […]
[00:03:43] zixy: That's what we actually need to validate.

(One turn per line, however long the turn runs; […] is where this example was cut.)

Enrolling runs the same diarization a transcript does and stores the pipeline's embedding for the speaker you name, so the vector is produced by exactly the model it will later be compared against. Whisper never runs — naming a voice does not need to know what it said. Enroll a person more than once (different mics, phone vs in-person) and matching takes their best sample, so a poor one dilutes nothing.

If the input has more than one speaker, --enroll refuses and lists them with their talking time; pick with --from-speaker N. Guessing would poison the store, and the damage would only show up as mislabelled transcripts later.

Voices live in ~/.config/transcribe/voices.json ($TRANSCRIBE_VOICES, or --voices). The store records which pipeline produced its embeddings and refuses to mix, because vectors from different models are not comparable.

Matching is cosine similarity against each enrolled voice, greedy from the strongest pair down, with each name used at most once — one person cannot hold both sides of a conversation. A speaker who matches nothing keeps its number, which is what should happen to a guest you have never enrolled.

Measured on this pipeline across three recordings of the same two people:

	cosine similarity
same voice, different recording	0.68 – 0.94
different voices	0.02 – 0.27

Nothing lands in between, so the 0.55 default sits in empty space. Raise it if you get false names, lower it if a known voice keeps coming out numbered.

Install

nix

No flakes; plain nix-build and nix-shell.

nix-build                             # the CLI, with diarization
nix-build --arg withDiarization false # transcription only, no torch
nix-build -A passthru.tests.merge     # just the unit tests
./result/bin/transcribe -d interview.mp4 -

The wrapper bakes ffmpeg and whisper-cpp into the binary's PATH, so ./result runs on a machine with nothing else installed. --arg withDiarization false drops pyannote for a ~50 MB closure instead of ~2 GB, when you never need -d.

nixpkgs is pinned in nix/nixpkgs.nix. To use your own channel instead:

nix-build --arg pkgs 'import <nixpkgs> { }'

Development shell — ffmpeg, whisper-cli and a python with pyannote, src/ on PYTHONPATH:

nix-shell

uv

uv venv --python 3.12 .venv
uv pip install -e '.[diarize]'

whisper-cli and ffmpeg have to be on PATH (or point WHISPER_CLI at a build).

Model access

The default pipeline, pyannote/speaker-diarization-community-1, is gated. Once, before the first -d run:

1. accept its terms at https://huggingface.co/pyannote/speaker-diarization-community-1
2. cache a read token with hf auth login

Weights are fetched on first use into ~/.cache/huggingface, never at build time. To check the grant took:

curl -so /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $(cat ~/.cache/huggingface/token)" \
  https://huggingface.co/pyannote/speaker-diarization-community-1/resolve/main/config.yaml

200 is good, 403 means the terms are not accepted (or the token is fine-grained without "read access to public gated repos").

Note that pyannote.audio 4.x loads its PLDA from community-1 whatever --pipeline you ask for, so this grant is required even to run the older pyannote/speaker-diarization-3.1. With it in place, either pipeline works — 3.1 needs no extra access beyond its own terms, which are likely already accepted if you used pyannote before.

The 3.x line of pyannote.audio also runs 3.1, without needing community-1 at all, but it caps torch below 2.9 and huggingface-hub below 1.0 — outside the binary cache and outside nixpkgs. That is why this targets 4.x.

Diarization prefers the pipeline's exclusive output, which resolves overlapping speech to one speaker per instant — the right input for aligning text to turns. On 3.x, where that does not exist, the plain annotation is used.

Platforms

Linux and macOS, x86_64 and aarch64. Nothing in the code is platform-specific — it shells out to ffmpeg and whisper-cli and reads wav with the stdlib — and the derivation evaluates for x86_64-linux, aarch64-linux and both darwins.

GPUs are where the platforms actually differ. whisper-cpp takes build flags:

# CUDA whisper on linux (cudaPackages is unfree)
NIXPKGS_ALLOW_UNFREE=1 nix-build -E 'with import ./nix/nixpkgs.nix { config.allowUnfree = true; };
  callPackage ./package.nix { whisper-cpp = whisper-cpp.override { cudaSupport = true; }; }'

vulkanSupport and rocmSupport are there too; on darwin metalSupport is already on. Diarization is a separate question: nixpkgs' torch is CPU-only unless the whole package set is built with config.cudaSupport = true, which is a large source build rather than a flag. So GPU diarization is close to free on a Mac and a project on Linux. auto degrades correctly either way: with a CPU-only torch, torch.cuda.is_available() is false and it picks cpu.

Speed

--device defaults to auto: cuda if torch sees it, else mps on Apple Silicon, else cpu. Measured on a 17-minute file, M-series, speaker-diarization-community-1:

device	diarization	output
mps	33 s	—
cpu	6 m 54 s	byte-identical

12× for free, so it is on by default. --device cpu is the escape hatch if a pipeline ever misbehaves on Metal; PYTORCH_ENABLE_MPS_FALLBACK=1 is set for you when mps is chosen, so an op Metal lacks drops to cpu instead of crashing.

whisper.cpp is the fast half either way — seconds on Metal.

Tests

pytest

covers the overlap assignment and turn merging, which is where the interesting mistakes live. It needs no torch, no models and no audio.

Known rough edge

A single whisper segment can span a speaker change, and max-overlap gives the whole segment to one of them. whisper's -oj output carries token-level timestamps, so the fix is to split such segments at the pyannote turn boundary; not done yet.