v0.1.0

A self-hosted voice agent runtime. Speech-to-text, the language model and text-to-speech run together in one process on hardware you own, streaming into each other, so a reply starts playing while it's still being generated. No per-minute API bills, no audio leaving your machine.

pip install fusion-runtime
frun models pull
frun up

Then open http://localhost:8000 and click Talk. You can talk over the agent to interrupt it.

Measured on an RTX 3090

The production profile exactly as it ships — Qwen 7B q4, Whisper small, Kokoro, all on one card.

	Median
Processing — turn ends, audio comes back	~490 ms
Stopwatch from your last syllable	991 ms
↳ of which: silence wait before the turn is judged over	~500 ms, configurable
Speech-to-text	119 ms
LLM first token	27 ms
Text-to-speech real-time factor	0.09
LLM tokens/sec	127

Two numbers because there are two honest answers. A stopwatch from your last syllable reads 991 ms, but half of that is the runtime waiting through silence to decide you've finished — which elapses while you're still finishing. Quote whichever you like; say which one.

Several callers at once

Callers	Response, median
1	~460 ms
4	~740 ms
8	~4600 ms

Four simultaneous callers land in the same range as one. Past that it saturates ‚Äî throughput plateaus near 0.7 turns/sec. The bottleneck is a single in-process llama.cpp context: at twelve callers the model's first token is 4790 ms of a 5312 ms response, while speech-to-text holds at 76 ms. Point the LLM at vLLM or llama-server to go further.

Not a fit yet

• More than four callers on one GPU without moving the LLM out
• Telephony ‚Äî not built
• Long calls ‚Äî genuinely unmeasured
• Non-English speech output ‚Äî transcription is multilingual, the five bundled voices are English

Links

• Docs: https://fusion-runtime.dev/docs
• PyPI: https://pypi.org/project/fusion-runtime/

Apache-2.0. Embed it in a commercial product, rebrand it, ship it closed.