Repository navigation
Releases
v0.1.4
Added
- Website: brisvox.com, with the benchmarks, a comparison with Pipecat and LiveKit Agents, how a call works, a quick start and an FAQ. The package homepages on crates.io, PyPI and npm point to it.
- Pause, then decide on barge-ins, on by default in agent files (
[interruption] pause_then_decide,falseto turn it off; off withpreset = "off"and in the engine library's own defaults) pauses
the bot's playout the moment a barge-in candidate appears (the caller hears silence as early as an interruption would give it) and, within
decide_ms(300 to 1750, default 1500), either interrupts as before or resumes the same response from the paused point: a cough, a
door, a keyboard or a "mm-hmm" no longer cuts the bot off for good. The default rule (rule = "words", the behaviour of LiveKit's
resume_false_interruption) interrupts at once on any recognised word of the caller or a command word ("stop", "wait", "dur", "bekle", ...), and resumes
when the speech has stopped and no word came; the bot's own echo does not count. Everything uncertain interrupts: speech that is still going at the end of
the window, a full hold buffer, a pause or a resume the transport cannot apply.rule = "never"leaves the decision to the guards and only resumes a
candidate they drop. Aftertighten_afterresumed pauses intighten_window_ms(default 2 in 60 s) the bot stops pausing on voice activity alone and waits for one
recognised word. Works on every transport (a pause is applied like a clear, and the audio the client had buffered and not played is sent again on resume), with limits: over WebRTC
and SIP a clear does not flush the client, so packets already sent play out and only the audio still in the server's own queue is sent again, and an RTVI client sees its
interruption events at a pause. The LLM context only ever holds what was played. New metricsbrisvox_barge_in_pauses_total{outcome}and
brisvox_barge_in_pause_seconds. Turning it on also turns on the echo recognition.pause_after_ms(default 400) delays the pause until the speech has lasted that long, so a click or a door bang never pauses the bot. Measured with the built-in harness
(bench/results/barge-in-noise.md,brisvox-bench denoise realistic --set fair; assumed recognizer miss rates, mostly synthetic noise, no live transport): noise without speech cut the bot off on 13 % (wideband) and 19 % (telephone) of the candidates it made, driven by synthetic voiced coughs (the real recordings made 4 and 8
candidates), with thewordsrule on none (2 % if the recognizer writes a word for a tenth of the noise windows), while the real barge-ins the recognizer misses are lost: weighted 1.2 % (wideband) and 5.1 % (telephone) when short utterances are missed twice as often as sentences, 0.7 % and 2.7 % at the sentence rate, 0 with a recognizer that hears every word
(today's guards decide on duration and lose none; a LiveKit-style configuration loses the same as this one, amin_words = 1interruption several times more). It is on by default as in LiveKit: a hands-on check over WebRTC in Turkish with a real recognizer (knocks and coughs did not stop the bot, "dur" and full sentences did, quiet background speech did not) matched the measurement; turn it off for a telephone agent whose callers interrupt with single short words.
A second person who talks still interrupts the bot. Seedocs/CONFIG.mdanddocs/ARCHITECTURE.md. - The short-command list that keeps "stop" and "wait" from being taken for the bot's echo has the Turkish "dur", "bekle", "yeter", "sus", "pardon", "dinle" and "hayır"
(case-insensitive, the Turkish dotted and dotless i folded). A candidate of at most two words with one of them is never held as the echo, also withpause_then_decideoff: it changes whatecho_text_matchholds. brisvox-bench denoise realistic: barge-in calls with background noise of many kinds (real recordings, synthetic cough, keyboard, door, dishes, music, street, babble),
second voices through a loudspeaker and a measured room, a modelled recognizer and per-candidate metrics, with committed fixtures and notices (fixtures/realistic,fixtures/callers).
v0.1.3
Added
-
Noise suppression of the caller's audio, opt in per agent:
[audio] noise_suppression = "rnnoise"
(default"off") filters the audio before the voice activity detector and the turn detector hear it,
with RNNoise through the pure-Rustnnnoiselesscrate (BSD-3-Clause; cargo featuredenoise-rnnoise,
not in the default features ofbrisvox-server; a server built without it refuses the file at the key's position).
The filter runs on the inference pool inside the VAD batch job, costs about 0.35 % of one core per
call (0.45 % on an 8 kHz call) and adds 20 to 25 ms of delay to what the VAD hears; it never delays or
drops the caller's audio, and a failing or overloaded filter lets the audio through unfiltered.
denoise_stt = truealso sends the filtered audio to the speech-to-text provider (defaultfalse:
independent studies found that denoised audio often raises the word error rate). Library:
brisvox_inference::denoise(the streamingDenoisertrait,VadStream::with_denoiser),
SessionBuilder::with_denoise,DenoiseStatsSnapshot. A call below a filter's minimum sample rate
runs without it (logged once), and a filter that cannot be built leaves the call with the plain audio and
a warning; both are counted inbrisvox_engine_events_total{event="denoise_bypassed_narrowband" | "denoise_build_failures" | "denoise_errors" | "denoise_feed_dropped"}. Measured with 34 real noise recordings
(bench/results/denoise-*.md): it does no harm on clean audio but did not reduce false speech starts
on wideband audio and shifted Smart Turn verdicts on telephone audio, so it stays off by default and is
meant to be tested on one's own calls.brisvox-bench denoise cpu|quality|transcribemeasures the CPU
per call, the false barge-ins and turn verdicts under licensed noise (fixtures/noise, public domain),
and streams the same mixes to a sherpa-onnx server for the word error rate. -
Reload the agent file without a restart. Edit the agent file and
POST /admin/reloadon
the health listener (orkill -HUPthe server on Unix, or setBRISVOX_AGENT_RELOAD_POLL_MSto
have the file polled for changes): new calls run the new definition, calls in progress keep
theirs until they end. The new file is validated like the start validates it, MCP servers
included; an invalid file, an MCP server that cannot be reached or a change that needs a restart
(the[recording]table, a provider limit or Smart Turn model the start did not set up) leaves
the old definition running and is reported with the file position. The MCP servers and provider
connections of the old definition close when its last call ends. The endpoint answers JSON and
needsBRISVOX_ADMIN_TOKEN(a separate secret from the metrics token) unless the health
listener is on loopback; off a loopback address without a token it is off. The file poll asks
for a reload once per change, so a refused file is not retried every interval. The server now
handlesSIGHUP(a closed terminal reloads instead of ending it). New metrics
brisvox_agent_reloads_total{result}andbrisvox_agent_generation, one log line per reload.
Seedocs/OPERATIONS.md("Reloading the agent") anddocs/CONFIG.md. For embedders, the new
brisvox_server::reloadmodule has the pieces (SwapRunner,RunnerRecipe,AgentSet,
ReloadHub). -
Language switching mid-call. An agent file can list several languages with
[languages]and
[[languages.speak]](a voice, and optionally a TTS model, for each). The speech-to-text then
listens to all of them for the whole call (Deepgramlanguage=multi, AssemblyAI language
detection, Gladia code switching, OpenAIlanguages), and when the caller speaks another
listed language on two utterances in a row (configurable, or once when the provider is
confident) the next reply is in that language, with that language's voice, without a new call.
The language comes from the STT provider's report (newlanguagefield on
SttEvent::Transcript) or, when it reports none, from a text detector. The reply language is
pinned per turn with aReply in <language>.line, never in the middle of a response. A file
that lists a language its TTS provider cannot speak (Deepgram Aura, Kokoro and Turkish) is
refused at its position. New metricbrisvox_language_switches_total, new agent protocol
eventlanguage.switched, newTtsConfig::languageandTtsConfig::modelfor adapters,
exampleexamples/agents/multilingual-tr-en.toml. Seedocs/CONFIG.md, "Languages".
Changed
-
Cheaper turn detection is the default: the Smart Turn model runs with each GELU fused into one
operation (BRISVOX_SMART_TURN_GRAPH=fused; about 40% less memory traffic per analysis and more
calls per core under load). In 0.1.2 it was opt-in because one early verdict on a test cut ended a
turn the published graph did not; measured on the official test set, both graphs do that at the same
small rate and end the same number of turns too soon (4000 clips with VNNI, 1000 without;
bench/results/smart-turn-early-verdict-2026-10-31.md).BRISVOX_SMART_TURN_GRAPH=originalruns
the file as published. -
Adapter API (library users):
SttEvent::Transcripthas a new fieldlanguage: Option<SpokenLanguage>(the language the STT provider reported for a final segment;Nonewhen
it reports none), andTtsConfighas two new fields,languageandmodel(the language and
TTS model a session should speak,Noneto keep the adapter's own). Code that builds these
values with struct literals must add the fields; code that matchesSttEvent::Transcript
without..must name the new field. The agent protocol gains thelanguage.switchedevent
(newLanguageSwitchedmessage in the Python and TypeScript SDKs). -
Background speech and echo handling, measured against the engine's real barge-in guards
(bench/results/noise-handling-calls.md,docs/LATENCY.md, "Noise handling beyond a denoiser"):[audio] noise_suppression = "hush"(experimental): Hush (Apache-2.0, pure Rust, trained for voice
agents with competing speakers; cargo featuredenoise-hush, not in the default features) as the filter in
front of the VAD and the turn detector: 1 % of a core per call, 10 ms of delay; takes a second voice 15 to
20 dB under the caller and the bot's echo out of the barge-ins in many calls, no extra delay for a real
barge-in. Its gain depends on the input level (it removes the speech of a quiet caller and distorts 8 kHz
phone audio), so a call below 16 kHz runs without it (logged, counted) and the agent file refuses it with
call_sample_ratebelow 16000.brisvox-bench denoise safetychecks that a filter leaves clean speech at
three caller levels alone."rnnoise"measured worse than nothing for the VAD and is not in the default
features either.[interruption] echo_text_match: holds a barge-in whose words are the bot's own, just said (the bot's
voice coming back); needsmin_words >= 1. Removed the echo case completely in the measurements. A caller who repeats
the bot's words back while it speaks is held like the echo until other words appear (the texts are the
same); short commands (at most two words with "stop", "wait", "no", ...) always get through.[vad] level_gate_db: speech may only start within that many dB of the caller's own level, so a quieter
second voice cannot start a barge-in.- Library:
brisvox_inference::Gated(level gates and a noise-floor estimate over any speech model),
brisvox_inference::denoise::blend::Blended(dry/wet mix for any denoiser),
brisvox_agent::{TurnMachine::set_echo, TurnMachine::set_interruption, echoes_bot, is_bot_echo, NoiseAdapt}.
brisvox-bench denoise callsdrives the real turn machine and policy over calls with background
speech (LibriSpeech,fixtures/interferers), bot echo and noise.
Fixed
- tokio 1.53.3 in the lock file (RUSTSEC-2026-0386,
AsyncFd::newand related constructors violate
I/O safety). brisvox does not call those constructors; the crates take any tokio 1.x, so their users
pick up the fix withcargo update -p tokio.
v0.1.2
Added
- Keyless local stack:
docker compose -f deploy/local/compose.yaml upstarts a voice agent
you talk to in the browser with no API key and no cloud: brisvox-server, a sherpa-onnx streaming
recognizer, Ollama withqwen2.5:1.5band a Kokoro-FastAPI voice, all on the CPU, with health
checks, a first-start model pull, a Llama 3.2 3B variant for faster machines and a
build-from-source override. A weekly workflow plays one recorded conversation at the stack. See
deploy/local/README.mdanddocs/DEPLOY.md. - MCP tools for agents: an agent file can name MCP (Model Context Protocol) servers, as a
program the server starts (stdio) or a streamable HTTP endpoint, with[[mcp.servers]]. The
tools on a server's allowlist are offered to the LLM like the server's own tools, with the same
guarantees: tools markedread_onlyin the file may run speculatively and are dropped on a
barge-in, every other tool never runs speculatively and never twice for one call,confirm
holds a tool for the caller's confirmation, and the engine's idempotency key travels in the
request's_meta. A stdio server that exits is restarted with backoff, and calls fail at once
meanwhile; the child process gets only its own environment, never the provider keys. Plain
http://is accepted only to a private-network host unless the entry setsallow_insecure, and
max_in_flight(default 32) bounds the calls in flight per server. Cargo
featuremcp(in the prebuilt binaries and the image). Seedocs/TOOLS.md("MCP tools"),
docs/CONFIG.mdand the exampleexamples/agents/mcp/docs-en.toml. - Grafana dashboard for the server's Prometheus metrics:
deploy/grafana/brisvox.json, one row
per operator question (healthy now, fast, do barge-ins work, why slow or refusing, providers
failing, cost). Tested with Grafana 12.1 (UI import and file provisioning). A test checks every
metric, label and label value it queries against the real/metricsoutput, and rejects
quantiles withoutleand counters without a rate. - Cheaper turn detection, opt-in:
BRISVOX_SMART_TURN_GRAPH=fusedruns the Smart Turn model
with each GELU fused into one operation (rewritten in memory from the pinned file): about 40% less
memory traffic per analysis and more calls per core under load; on 4000 clips of the official
Smart Turn v3.2 test set the verdicts matched the published graph on 3997 and the accuracy did not
drop. It is not the default because on one abrupt cut of the test fixtures its early verdict ended
a turn that the published graph's did not (docs/providers/smart-turn.md, "Fused GELU").
Changed
- Benchmarks: the capacity numbers in the README and the book now come from cloud containers
(AMD EPYC 9655P, 4 vCPU / 4 GB and 8 vCPU / 8 GB) with every call going through each system's own
provider client code to protocol-faithful fake providers over TLS, and the real Silero VAD and
Smart Turn models: brisvox 250 and 470 calls within the latency SLO against 1 and 10 for
Pipecat 1.12.0, and 430 and 780 without a failed call or lost turn against 10 and 10. The local
harness runs the same comparison:PROVIDERS=fakes MODELS=1 bench/capacity_linux.sh.
Fixed
- TLS certificate checks: rustls-webpki 0.103.16 and webpki-roots / webpki-root-certs 1.0.11, so
provider connections no longer trust the distrusted emSign intermediates and reject distrusted
intermediates in a chain (RUSTSEC-2026-0381, RUSTSEC-2026-0382, RUSTSEC-2026-0383). - sherpa-onnx: a hang-up ends the stream with
Doneand lets the server close the
socket, so the Python server no longer logs a failed connection after every call.
v0.1.1
Added
- Usage and cost per call: the engine counts the provider billing units of every call (LLM
tokens including discarded speculative responses, STT audio seconds sent, TTS characters sent)
and the server delivers one per-call summary (duration, turns, average and worst stage latency,
units with the configured provider and model) as the agent link eventsession.usagebefore
session.end, as thesummaryof the transcript'sendline, as onecall summarylog line
and as the Prometheus familiesbrisvox_llm_tokens_total,brisvox_stt_audio_seconds_total,
brisvox_tts_characters_totalandbrisvox_call_duration_seconds;bot.llm_stoppedcarries
the tokens of each turn, and the Python and TypeScript agent SDKs have a typedsession.usage
event. Units, not money (docs/OPERATIONS.md, "Usage and cost"). - Learned admission ceiling (
BRISVOX_OVERLOAD_CEILING, on by default). Pool pressure shows
only when callers speak, so a fast ramp used to be admitted far past capacity (on 4 vCPU, about
730 calls held for a capacity of about 250, every call at 3.7 s p50). The server now learns how
many calls it serves well from the pressure signals and refuses new calls (503) above that,
without touching running calls. In a load test with 900 callers dialing into 4 vCPU: 120 to 400
calls held at 0.9 to 1.5 s p95, none failed. - Cautious start: a fresh server admits 16 calls and doubles per calm window until the pool is
first under pressure, so a rush right after a restart cannot overshoot. - A second agent per server (
BRISVOX_WS_AGENT_FILE): the generic websocket listener runs its
own agent, sharing the process's pools and admission control. BRISVOX_DEFAULT_AGENT_FILE: the container image's default agent path, used only when neither
BRISVOX_AGENT_FILEnorBRISVOX_AGENT_DEFINITIONis set.
Changed
- Library API (
brisvox-engine,brisvox-agent-link,brisvox-server):EngineEvent::BotLlmStopped
has a newusagefield (patterns need..);CallGuard::newtakes asummarizerargument;
Metrics::observe_callandCallObserver::call_endedtake asummaryargument;
observability::metrics::Histogram::newis no longerconst(useHistogram::with_boundsfor
other buckets); newAgentLink::close_with(closeis unchanged). BRISVOX_MAX_SESSIONSdefaults to 256 per core (at least 256) instead of 256.- Pool pressure also watches the Smart Turn executor (its queue waits): a late end-of-turn verdict
delays the reply as much as a late VAD window./readyzreportsoverloadedat the learned
ceiling and when the Smart Turn executor is behind, so a load balancer sends calls elsewhere.
Fixed
- An admitted call could stay silent on a saturated pool: the output resampler is now built off
the pool when the pool is full or late. - The container image set
BRISVOX_AGENT_FILE, soBRISVOX_AGENT_DEFINITIONalone (the way to
pass an agent on a platform that cannot mount a file) failed at startup. - The release workflow's npm publish step (
./pkg/*.tgz): 0.1.0 reached GitHub, GHCR and PyPI
only; crates.io and npm start at 0.1.1.
v0.1.0
First public release.
Added
- Engine. A Rust-native real-time voice pipeline, one isolated session per call: transport
pump, VAD, STT, turn driver, LLM runner, TTS and transport out. Turn epochs drop stale frames
everywhere, so an interruption never leaves old audio or text behind. After an interruption the
LLM context holds only what was actually played (played-text truncation). Every channel is
bounded, every network await has a timeout, every task is cancellable, and one session's panic or
provider failure cannot touch another. - Server.
brisvox-server, a multi-tenant session server: one TOML or YAML file per agent
(brisvox-server check <file>validates it offline with line and column for every problem),
health and readiness endpoints, Prometheus metrics, OpenTelemetry traces (featureotlp),
admission control, graceful drain, recordings encrypted at rest with age (feature
recording-encryption), a container image and a compose file. - Transports. Generic websocket, Twilio Media Streams, Telnyx media streaming, RTVI (Pipecat's
client SDKs), WebRTC (featurewebrtc, WHIP-style signaling, Opus) and a SIP trunk listener
(featuresip). G.711 mu-law and A-law, any call rate with any provider rate. - Providers. STT: Deepgram, AssemblyAI, Gladia, sherpa-onnx (on your machine) and servers that
speak the OpenAI Realtime transcription protocol. LLM: OpenAI, Anthropic, Gemini and any
OpenAI-compatible endpoint (Groq, Together, Fireworks, vLLM, Ollama, OpenRouter). TTS: Cartesia,
ElevenLabs, Deepgram Aura and any server of the OpenAI/v1/audio/speechshape. Speech-to-speech:
OpenAI Realtime and Gemini Live. Per-stage failover, circuit breakers and retry budgets. - On-device models. Silero VAD (compiled into the binary, MIT) and Smart Turn v3.2 end-of-turn
detection on ONNX Runtime (featuresmart-turn-ort), on a dedicated inference pool. - Conversation. Adaptive turn taking, interruption policy with backchannel lists, speculative
LLM start, tool calls withread_onlyandside_effectkinds (a side-effect tool never runs
speculatively and never runs twice), filler phrases for slow tools, DTMF in and out, call
transfer, answering machine handling. English and Turkish. - SDKs. The Python package
brisvox-agentand the npm package@brisvox/agentfor an
out-of-process agent (your tools and application logic in your own process, the server runs the
call), and@brisvox/web, a dependency-free browser module for a "Talk to us" button: WebRTC
first, automatic WebSocket fallback, microphone, playback, barge-in and clean hang-up. - Rust crates.
brisvox-core,brisvox-audio,brisvox-inference,brisvox-agent,
brisvox-providers,brisvox-sim,brisvox-transport,brisvox-engine,brisvox-agent-link
andbrisvox-serveron crates.io;brisvox_server::app::run_with_toolsis a wholemainfor a
server with your own Rust tools. - Umbrella packages. The plain name
brisvoxis a real package on all three registries, so
cargo add brisvox,pip install brisvoxandnpm i brisvoxwork next to the specific names.
The crate re-exportsbrisvox-server, the tools API (brisvox-agent), the provider adapters and
the core types and forwards the server's Cargo features (brisvox::run_with_tools(...)is a
wholemain). The PyPI package depends onbrisvox-agentof the same release and re-exports its
API (from brisvox import Agent). The npm package depends on@brisvox/agentand@brisvox/web
and re-exports them (brisvoxis the agent SDK for Node,brisvox/webthe browser client). - Benchmarks. A harness for voice-to-voice latency and capacity with real and simulated
providers, with the raw results and the method committed underbench/. - Documentation. The book (architecture, quickstart, configuration, telephony, operations,
benchmarks, protocols and provider notes) andllms.txt/llms-full.txtfor AI coding agents. - Release. Prebuilt
brisvox-serverbinaries for Linux (x86_64, aarch64), macOS (aarch64)
and Windows (x86_64) with checksums, and a multi-architecture image at
ghcr.io/bilal07karadeniz/brisvox-server.