Repository navigation
RealtimeVAD 0.1.0
The product is the early pause: a median 92 ms before Silero notices that the speaker stopped. end is now exactly Silero's end, and pause got more accurate at the cost of about 8 ms of lead.
pip install -U realtimevadNumbers and tables: README, docs/results.md.
Changed behaviour
endcomes where Silero alone ends the segment. Up to 0.0.5 the quiet clock could end a segment earlier. That is now the optionVAD(early_end=True)(also onVADIterator);Segment.earlysays whether the quiet clock made the end. If you relied on the earlierend, set it.pauseis repeated. When sound comes back after apauseand the audio goes quiet again, a newpausefollows. The lastpauseis more than 100 ms too early in 1 to 9 % of clips instead of 6 to 11 %; its median lead over Silero is 92 ms instead of 100 ms on the one-speaker turns.VAD(repause_s=None)gives onepauseper stretch of speech, as before. Sound in the trailing silence does not repeat apause.
New
- A short first word is kept with a long
start_s. Withstart_sraised to reject clicks (the demo uses 0.32 s), a short "Hey" before more speech did not last long enough and the segment began at the second word. The start now reaches back to speech of 64 ms or more that stopped less thanend_sbefore. A short sound that stands alone still starts nothing. Nothing changes with the defaultstart_s. pause_mode:"accurate"(default),"balanced"or"fast"."fast"leads Silero by 118 to 130 ms and puts about one in five last pauses inside the last word.resumeevent (VAD(resume_events=True)): the speech went on after apause; cancel what you started.- Steady noise close under the speaker is subtracted before the quiet check (
noise_subtract_db). A noisy quiet line and a quiet phone line had no lead in 0.0.5 and now lead by 84 to 114 ms (simulated noise and line). - LiveKit plugin, now on PyPI:
pip install livekit-plugins-realtimevad(0.1.0).realtimevad.VAD.load()in place ofsilero.VAD.load(), andSpeculativeSTTstarts LiveKit's preemptive generation atpause. A community plugin, not made by LiveKit. See its README.
Fixed
- A DC offset in the input no longer hides the silence from the quiet clock (levels are measured behind a 20 Hz high-pass).
- The "Silero ignores speech-loud audio" recovery could restore a state from inside the previous segment and report a phantom segment.
- With the early end, the recovery could run on other frames than without it, and one clip of 7,080 ended later than Silero alone.
- With the early end, a segment that opened right after an early end could end up to three frames before Silero's own count.
- Demo page: the waveform was drawn later than the detector's events by the time the connection took to open, so a pause could look as if it cut into the speech.
Tools and demo
benchmarks/long_stream.py: turns in one long stream against a fresh detector per turn. No drift on 72 minutes of one-speaker turns (results); a test now guards it.python demo/server.py --record DIRkeeps a demo session's audio and events in local files, so a session that looked wrong can be replayed.- The demo page now shows the last pause against Silero as its headline, and the demo video was recorded again with 0.1.0.
Removed
- No Pipecat integration is planned: its speech recognizers finalize only when the VAD reports the stop, so nothing can start at
pause(issue 4).
Known
- A reported drift of
pausein long sessions did not reproduce: 72 minutes as one stream gave the same pauses as a fresh detector per turn. benchmarks/long_stream.pyshows no lead for about half the turns when it is given a recorded demo session. That is a weakness of the tool, not of the detector.
Limits
Alpha. The lead needs a close microphone; on a distant one there is none, and RealtimeVAD then behaves like Silero alone. See the README.
Files
| File | SHA-256 |
|---|---|
realtimevad-0.1.0-py3-none-any.whl |
7b3123df0925c174c2b0e725cb00f0f825ddb05a5335cddb557682000df84350 |
realtimevad-0.1.0.tar.gz |
3e4fbb31aea6876c0541958fa03a7339ba664f4b8e3c239eca01071f47799344 |