Skip to content

RealtimeVAD 0.0.5

Choose a tag to compare

@KoljaB KoljaB released this 12 Nov 19:31
· 63 commits to main since this release

Quiet speakers are no longer cut, short "mm-hmm"s are heard, and the end now beats Silero's in 82 % of turns instead of 51 %.

pip install -U realtimevad

Fixed

  • Soft word endings cut on quiet input (#1). "Quiet" had a fixed floor at about −54 dBFS. For a quiet speaker (a phone line at source level) that floor sat a few dB under the speech, and the segment ended while they were still talking. The floor now follows the speaker: quiet_rms or 20 dB below their own level, whichever is lower (floor_margin_db). On 3,401 real turns at −52 dBFS, turns that lost 100 ms of speech or more went from 868 to 26 (Silero alone: 23).
  • Short backchannels not reported as speech (#2). Silero scores many short hums between 0.5 and 0.9, below the start threshold. Such speech now starts a segment after about 0.13 s at 0.5, if it is clearly louder than the room (soft_start_s). Of 200 real backchannels, 175 are reported instead of 150; Silero itself stays below 0.5 for 15.
  • A click in the quiet after a word stopped the early end. One loud 10 ms block (a click, a lip smack, a breath) left the end to Silero. The quiet count now starts again after it. end is earlier than Silero alone in 82 % of turns (was 51 %), by a median 70 ms (was 18 ms).
  • "Never later than Silero alone" is exact now. Silero's state recovery comes on the same frame with and without the quiet clock.

New

  • normalize_to_dbfs (off by default): amplifies what Silero sees for quiet or phone audio. At −62 dBFS, turns that lost 100 ms or more went from 419 to 40. It has a small cost at normal level, see the README.
  • A LiveKit Agents plugin in the repository (not on PyPI): realtimevad.VAD.load() in place of silero.VAD.load(), with measurements against the Silero plugin on the same audio.
  • An edge-case test suite on about 100 MB of freely licensed audio (8 languages, noise, music, coughs, whisper, real backchannels), downloaded by script, not in git.
  • benchmarks/quiet_input.py to reproduce the quiet-input measurement on your own recordings.

Changed numbers

  • Speech-off lead: a median 100 ms before Silero's probability drops (was 110 ms), 206 ms before VADIterator's end, on 3,279 real turns. The 10 ms went into not cutting quiet speakers.
  • On a noisy quiet line (noise within about 20 dB of the speaker) there is no early end any more. It ends where Silero does.

Behaviour changes to know

  • More speech starts a segment (the soft start). Turn it off with VAD(soft_start_s=None).
  • The old fixed floor is VAD(floor_margin_db=None).

Limits

Alpha. The numbers come from one speaker and one capture chain; quiet and phone input were simulated from those recordings. See Quiet input and phone audio and Limits.

Files

File SHA-256
realtimevad-0.0.5-py3-none-any.whl a66a0731c0da8be31e8e47065f6d37e2ff706629b018575469bf5f77722f92a9
realtimevad-0.0.5.tar.gz f8babee8818932c150f46469cd4635616ce8abe0eede2427c0cf7891c19308ab