Separate Voice now hears the people in the room: it counts the speakers, shows
you the count before anything lands, and gives you one full-length track per
speaker plus Backing.
Added
- Separate Voice lands one track per speaker.
Pipeline → Separate Voice
is the same command, and when more than one person is talking it goes one
step further. It runs three stages instead of one: the existing HT-Demucs
separation, then speaker segmentation (pyannote-segmentation-3.0 over 10 s
windows shifted by 1 s —SEG_WINDOW/SEG_SHIFTin
electron/diarizeHost.cjs) and speaker embedding (WeSpeaker ResNet34-LM with
per-utterance mean subtraction) in a new utility process, then clustering and
assembly in the renderer. Then it stops: a confirmation step shows the
count it found, each speaker's speech time and share, and the memory a
landing would take. Nothing lands until you press Land, and changing the
count re-groups the measurements already taken — instantly, with no second
model run. Landing produces<name> — Speaker 1 … Speaker Nplus
<name> — Backing. A count of one lands exactly the Voice + Backing result it
always did. The two models are one 32.5 MB download (5,992,913 B
segmentation + 26,530,550 B embedder =DIARIZE_MODEL_BYTES32,523,463 in
src/services/diarizeService.ts), offered beside the 166 MB separation model.
The grouping policy is fixed and in code: threshold0.55, minimum cluster
size4, minimum speaker share5 %, at most6speakers
(DIARIZE_THRESHOLD/MIN_CLUSTER_SIZE/MIN_SPEAKER_SHAREin
src/dsp/diarization.ts,MAX_SPEAKERSinsrc/dsp/speakerClustering.ts).
New:src/dsp/diarization.ts,src/dsp/spanMask.ts,
electron/diarizeHost.cjs,electron/diarizeManager.cjs,
src/services/diarizeService.ts,scripts/diarize-bench.cjs,
scripts/fetch-diarization-assets.cjs; changed:
src/components/Dialogs/SeparateDialog.tsx,src/services/stemLanding.ts,
electron/main.cjs,electron/preload.cjs,src/App.tsx. - A speaker track is the whole voice with the other people silenced, and the
landing carries no exact-sum claim. Each speaker track is the full-length
voice stem with everything outside that speaker's turns taken to silence, and
every kept turn faded in and out over 10 ms so its edges do not click.
Where two people talk at once, that audio is written into both their
tracks. So speakers + Backing does not add back up to the source — unlike
Separate into Stems (bit-exact) and the one-voice landing (within float32
rounding, worst 4.32e-7). The dialog says so before you land and prints no
exactness note afterwards; the Backing on its own is unchanged and still adds
back. The full write-up is indocs/KNOWN_LIMITATIONS.mdunder A speaker
split is a COUNT that was right four times out of four, and nothing finer. - What the count rests on — and what it does not. From
docs/bench/diarize-bench-baseline.json(written by
scripts/diarize-bench.cjs): on the four test recordings the counts came out
2 / 2 / 2 / 4against a truth of2 / 2 / 2 / 4— 4 of 4 — in both
measured conditions,--direct(16 kHz speech straight into the speaker step)
and--full-chain(what the tool actually does: HT-Demucs first, then the
Vocals stem). Audio-anchored consistency was 100 / 100 / 96.5 / 100 % direct
and 100 / 91.9 / 96.8 / 100 % through the chain. That is the whole of the
evidence, and the same file says what it is not: four recordings, about
162 s in total; the truth shipped with them is a speaker count per
file and nothing finer, so there is no diarization error rate — a row can
have the right count with the wrong turns, and nothing here measures who spoke
when. One of the four is the only four-speaker case and it is Mandarin read
by an English-trained embedder (WeSpeaker VoxCeleb ResNet34-LM). Overlapping
speech is near-absent in all four, so overlap handling is untested end to
end, and recordings with many short turns or heavy crosstalk were not in the
set. The count is a control you can overrule for exactly that reason, and the
panel prints the limit beside it.
Changed
- The Separate Voice time estimate and its progress bar carry the measured
stage rates. Both seeds were re-derived from
docs/bench/diarize-bench-baseline.jsonon an idle machine — segmentation
10 ms and embedding 75 ms per audio second (MEASURED_SEGMENT_MS_PER_S
andMEASURED_EMBED_MS_PER_S,src/services/diarizeService.ts), replacing 8
and 55, which came from an early spike that had timed a different embedder
(CAM++, which this feature does not ship). The weighted bar now hands Demucs
88.6 % of the run instead of 91.3 % (stageWeights(), computed from those
two seeds andMEASURED_REALTIME_FACTOR= 1.52 in
src/services/stemService.ts). The estimate assumes an otherwise idle
machine. Its Demucs term is the conservative end of the app's own two stem
measurements, while this bench measured the stem stage at 1.97–2.07× realtime
(msPerAudioSecond.stem483.2–507.2 across the four full-chain rows), so on
an idle machine the panel reads long — the safe direction. On a busy one
it reads short:docs/KNOWN_LIMITATIONS.mdrecords the same bench run beside
a full test suite putting the chain at 1,150–1,230 ms per audio second, about
40 % over what the panel predicts. The per-stage spread through the chain
is in the baseline: segmentation 6.0–11.9 ms and embedding 39.9–110.1 ms per
audio second. Affects:src/services/diarizeService.ts. - The speaker landing's memory gate prices the Backing document too. A
landing allocates N speaker documents plus a full-length Backing, so the
panel quotes and refuses on (N + 1) × the document size. At 15 minutes of
44.1 kHz stereo one document is 317.5 MB, so two speakers cost 952.6 MB (not
635.0 MB) and three cost 1.3 GB; a landing above 1.2 GB
(SPEAKER_LANDING_BUDGET_BYTES,src/services/stemLanding.ts) is refused
with the figure in hand rather than landed short. None of these documents has
been written to disk, so a project save writes every one of them into the
project file — export the speakers you want and close the rest. Affects:
src/components/Dialogs/SeparateDialog.tsx,src/services/stemLanding.ts.
Verify your download (see SHA256SUMS.txt):
CertUtil -hashfile "Auditorium Setup 1.39.0.exe" SHA256