Repository navigation
Releases
RealtimeVAD 0.1.1
Quiet streams: any number of extra quiet detectors on one VAD, each with its own sensitivity. Nothing changes for existing code: without a stream the events are the same as in 0.1.0.
pip install -U realtimevadvad = VAD()
cut = vad.add_stream("cut", offset_db=6.0, quiet_s=0.3) # quiet 6 dB sooner than pause, held for 0.3 s
for event in vad.feed(chunk):
if event.kind == "quiet" and event.stream == "cut":
... # event.quiet.start, .duration
elif event.kind == "sound" and event.stream == "cut":
... # event.quiet.start, .end, .duration
cut.offset_db = 9.0 # any setting, any timeNew
- Quiet streams:
vad.add_stream(name, offset_db=..., quiet_s=..., rearm_s=...)adds one more quiet detector to the sameVAD: its own offset on the quiet thresholdpauseuses, its own minimum length and its own rule for what ends the quiet. A stream reportsquietandsoundevents with where the audio went quiet and for how long. It reuses the instance's levels and noise tracking, so Silero runs once.start,pause,resumeandendare identical with and without streams (20 minutes of meeting audio, event by event). Four streams cost about 0.5 ms per second of audio. README, design note. block_events=True: ablockevent for every 10 ms block with its level, the quiet threshold andover_db, how far the block is from counting as quiet, for callers who want their own rule.benchmarks/quiet_streams.pychecks the equality and measures the cost on your own WAVs.
RealtimeVAD 0.1.0
The product is the early pause: a median 92 ms before Silero notices that the speaker stopped. end is now exactly Silero's end, and pause got more accurate at the cost of about 8 ms of lead.
pip install -U realtimevadNumbers and tables: README, docs/results.md.
Changed behaviour
endcomes where Silero alone ends the segment. Up to 0.0.5 the quiet clock could end a segment earlier. That is now the optionVAD(early_end=True)(also onVADIterator);Segment.earlysays whether the quiet clock made the end. If you relied on the earlierend, set it.pauseis repeated. When sound comes back after apauseand the audio goes quiet again, a newpausefollows. The lastpauseis more than 100 ms too early in 1 to 9 % of clips instead of 6 to 11 %; its median lead over Silero is 92 ms instead of 100 ms on the one-speaker turns.VAD(repause_s=None)gives onepauseper stretch of speech, as before. Sound in the trailing silence does not repeat apause.
New
- A short first word is kept with a long
start_s. Withstart_sraised to reject clicks (the demo uses 0.32 s), a short "Hey" before more speech did not last long enough and the segment began at the second word. The start now reaches back to speech of 64 ms or more that stopped less thanend_sbefore. A short sound that stands alone still starts nothing. Nothing changes with the defaultstart_s. pause_mode:"accurate"(default),"balanced"or"fast"."fast"leads Silero by 118 to 130 ms and puts about one in five last pauses inside the last word.resumeevent (VAD(resume_events=True)): the speech went on after apause; cancel what you started.- Steady noise close under the speaker is subtracted before the quiet check (
noise_subtract_db). A noisy quiet line and a quiet phone line had no lead in 0.0.5 and now lead by 84 to 114 ms (simulated noise and line). - LiveKit plugin, now on PyPI:
pip install livekit-plugins-realtimevad(0.1.0).realtimevad.VAD.load()in place ofsilero.VAD.load(), andSpeculativeSTTstarts LiveKit's preemptive generation atpause. A community plugin, not made by LiveKit. See its README.
Fixed
- A DC offset in the input no longer hides the silence from the quiet clock (levels are measured behind a 20 Hz high-pass).
- The "Silero ignores speech-loud audio" recovery could restore a state from inside the previous segment and report a phantom segment.
- With the early end, the recovery could run on other frames than without it, and one clip of 7,080 ended later than Silero alone.
- With the early end, a segment that opened right after an early end could end up to three frames before Silero's own count.
- Demo page: the waveform was drawn later than the detector's events by the time the connection took to open, so a pause could look as if it cut into the speech.
Tools and demo
benchmarks/long_stream.py: turns in one long stream against a fresh detector per turn. No drift on 72 minutes of one-speaker turns (results); a test now guards it.python demo/server.py --record DIRkeeps a demo session's audio and events in local files, so a session that looked wrong can be replayed.- The demo page now shows the last pause against Silero as its headline, and the demo video was recorded again with 0.1.0.
Removed
- No Pipecat integration is planned: its speech recognizers finalize only when the VAD reports the stop, so nothing can start at
pause(issue 4).
Known
- A reported drift of
pausein long sessions did not reproduce: 72 minutes as one stream gave the same pauses as a fresh detector per turn. benchmarks/long_stream.pyshows no lead for about half the turns when it is given a recorded demo session. That is a weakness of the tool, not of the detector.
Limits
Alpha. The lead needs a close microphone; on a distant one there is none, and RealtimeVAD then behaves like Silero alone. See the README.
Files
| File | SHA-256 |
|---|---|
realtimevad-0.1.0-py3-none-any.whl |
7b3123df0925c174c2b0e725cb00f0f825ddb05a5335cddb557682000df84350 |
realtimevad-0.1.0.tar.gz |
3e4fbb31aea6876c0541958fa03a7339ba664f4b8e3c239eca01071f47799344 |
RealtimeVAD 0.0.5
Quiet speakers are no longer cut, short "mm-hmm"s are heard, and the end now beats Silero's in 82 % of turns instead of 51 %.
pip install -U realtimevadFixed
- Soft word endings cut on quiet input (#1). "Quiet" had a fixed floor at about −54 dBFS. For a quiet speaker (a phone line at source level) that floor sat a few dB under the speech, and the segment ended while they were still talking. The floor now follows the speaker:
quiet_rmsor 20 dB below their own level, whichever is lower (floor_margin_db). On 3,401 real turns at −52 dBFS, turns that lost 100 ms of speech or more went from 868 to 26 (Silero alone: 23). - Short backchannels not reported as speech (#2). Silero scores many short hums between 0.5 and 0.9, below the start threshold. Such speech now starts a segment after about 0.13 s at 0.5, if it is clearly louder than the room (
soft_start_s). Of 200 real backchannels, 175 are reported instead of 150; Silero itself stays below 0.5 for 15. - A click in the quiet after a word stopped the early end. One loud 10 ms block (a click, a lip smack, a breath) left the end to Silero. The quiet count now starts again after it.
endis earlier than Silero alone in 82 % of turns (was 51 %), by a median 70 ms (was 18 ms). - "Never later than Silero alone" is exact now. Silero's state recovery comes on the same frame with and without the quiet clock.
New
normalize_to_dbfs(off by default): amplifies what Silero sees for quiet or phone audio. At −62 dBFS, turns that lost 100 ms or more went from 419 to 40. It has a small cost at normal level, see the README.- A LiveKit Agents plugin in the repository (not on PyPI):
realtimevad.VAD.load()in place ofsilero.VAD.load(), with measurements against the Silero plugin on the same audio. - An edge-case test suite on about 100 MB of freely licensed audio (8 languages, noise, music, coughs, whisper, real backchannels), downloaded by script, not in git.
benchmarks/quiet_input.pyto reproduce the quiet-input measurement on your own recordings.
Changed numbers
- Speech-off lead: a median 100 ms before Silero's probability drops (was 110 ms), 206 ms before
VADIterator's end, on 3,279 real turns. The 10 ms went into not cutting quiet speakers. - On a noisy quiet line (noise within about 20 dB of the speaker) there is no early end any more. It ends where Silero does.
Behaviour changes to know
- More speech starts a segment (the soft start). Turn it off with
VAD(soft_start_s=None). - The old fixed floor is
VAD(floor_margin_db=None).
Limits
Alpha. The numbers come from one speaker and one capture chain; quiet and phone input were simulated from those recordings. See Quiet input and phone audio and Limits.
Files
| File | SHA-256 |
|---|---|
realtimevad-0.0.5-py3-none-any.whl |
a66a0731c0da8be31e8e47065f6d37e2ff706629b018575469bf5f77722f92a9 |
realtimevad-0.0.5.tar.gz |
f8babee8818932c150f46469cd4635616ce8abe0eede2427c0cf7891c19308ab |
RealtimeVAD 0.0.4
First public release of RealtimeVAD: know that the speaker stopped, about 110 ms before Silero does.
pip install realtimevadWhat it does
A voice assistant can't do anything until it knows the user has stopped talking. RealtimeVAD keeps Silero VAD to decide what is speech and adds a faster check: every 10 ms it compares the audio level with the room's noise. The moment the audio goes quiet, you get a pause event and can start speculative work (ASR finalization, the LLM call, TTS), then cancel it if speech continues.
- Lead: speech-off a median 110 ms before Silero's probability drops, and 214 ms before Silero's
VADIteratorreports the end, measured on 3,238 real voice-assistant turns. - Price: it also fires in short gaps between words, about ten times as often as Silero. Use
pauseonly for work you can cancel. - Events:
start,pause,end, and optionalonset. - Drop-in:
from RealtimeVAD import VADIteratorreplaces Silero'sVADIteratorwith the same constructor and results. - Runs on: one CPU thread. Needs only numpy and onnxruntime; the Silero model is bundled, no torch.
Also in the repository
- A live demo that shows, on your own microphone, how much sooner RealtimeVAD knew you stopped than Silero alone.
- SPEC.md with what the detector guarantees.
benchmarks/speech_off_lead.pyto reproduce the measurement on your own recordings.
Status
Alpha. The API may still change. The numbers come from one speaker and one capture chain with noise suppression; see Limits.
Files
| File | SHA-256 |
|---|---|
realtimevad-0.0.4-py3-none-any.whl |
8547c743b735a3a9e8aa80b6461a35f8dc85824898fc515d074cd7dc2bce8fbf |
realtimevad-0.0.4.tar.gz |
c580f9742ec2d5fde083308c9e3235da2ab0a40f2edc521e1fdf8e4fc1a2a9cc |