Skip to content

v1.0.0

Choose a tag to compare

@sansamour sansamour released this 17 Aug 03:57
· 5 commits to voicechat since this release

First release of the VoiceChat fork of llama.cpp. Speech in, speech out, one
duplex timeline, with tool calls.

Downloads

file what
llama-voicechat-bin-win-cpu-x64.zip Windows x64, CPU. Needs an AVX2 CPU (Haswell 2013 or newer).
llama-voicechat-bin-win-cuda-12.8-x64.zip Windows x64, CUDA 12.8. Self-contained.
cudart-llama-bin-win-cuda-12.8-x64.zip The CUDA runtime dlls. Unzip into the same folder as the CUDA build.

The CUDA build carries device code for sm_86, sm_89 and sm_120 (RTX 30xx, 40xx,
50xx) and PTX from sm_50 up, so anything Maxwell or newer runs.

Model

The four GGUF files are in the llamacpp/ folder of
hoidhxd/NVIDIA-NemotronLabs-VoiceChat-11B-GGUF:

hf download hoidhxd/NVIDIA-NemotronLabs-VoiceChat-11B-GGUF --include "llamacpp/*" --local-dir .
llama-voicechat -m llamacpp/nemotron_voicechat_11b-stt-llm-Q4_0.gguf --mmproj llamacpp/mmproj-voicechat-perception-Q4_0.gguf --tts llamacpp/voicechat-tts-Q4_0.gguf --audio question.wav --tts-out answer.wav

Set VC_NO_BARGE=1 and VC_FORCE_BOS=1 for anything push-to-talk.

What is in it

  • llama-voicechat: the 12.5 Hz duplex loop, the function channel for
    turn-taking and tool calls, --serve for a live session, --say to make a
    test clip without a mic
  • a new mtmd projector type for VoiceChat's causal FastConformer encoder
  • the speech generator: gemma3 backbone, mixture-of-Gaussians head, 31 stage
    residual VQ and the codec decoder, on plain ggml graphs
  • three converters that split the published VoiceChat GGUF into the four files
    llama.cpp loads, plus a numpy reference harness for the speech path

The model answers in English only.

Everything upstream llama.cpp does still works; this is a fork of
ggml-org/llama.cpp with its full history.