FunASR llama.cpp runtime v0.1.9

Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime, with SenseVoice, Paraformer, and Fun-ASR-Nano support plus built-in FSMN-VAD.

This release moves the SenseVoice Latest Release source snapshot onto current main, including the yaml.SafeLoader security fix from #312. Users downloading GitHub's generated source archives no longer receive the older unsafe YAML loader from runtime-llamacpp-v0.1.4.

Choose an asset

• Default linux-x64 / windows-x64: maximum CPU compatibility.
• linux-x64-avx2 / windows-x64-avx2: higher throughput on CPUs with AVX2, FMA, F16C, and BMI2.
• linux-x64-vulkan / windows-x64-vulkan: Vulkan backend; requires a working Vulkan driver and ICD.
• windows-x64-cuda: CUDA backend targeting architecture 86; build from source for other GPU architectures.
• Native linux-arm64 and macos-arm64 packages are included.

Download a default quantized model with:

pip install -U huggingface_hub
bash download-funasr-model.sh sensevoice

Then run llama-funasr-sensevoice; no Python ASR runtime or local compilation is required.

All nine files are byte-for-byte identical to the digest-verified assets from modelscope/FunASR runtime v0.1.9.

Documentation: https://github.com/QwenAudio/SenseVoice/blob/runtime-llamacpp-v0.1.9/runtime/llama.cpp/README.md

FunASR llama.cpp runtime v0.1.4

Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime.

This release keeps the default x64 assets conservative for maximum CPU compatibility and adds explicit x64-avx2 assets for CPUs with AVX2/FMA/F16C/BMI2. If v0.1.3 fixed SIGILL but was too slow on your machine, try funasr-llamacpp-windows-x64-avx2.zip.

Use the default x64 package if the AVX2 package fails with Illegal instruction / SIGILL.

FunASR llama.cpp runtime v0.1.3

Portable rebuild of the FunASR llama.cpp / GGUF runtime binaries.

This release rebuilds the v0.1.2 runtime with conservative CPU flags for the generic x64 assets. In particular, it disables ggml native CPU tuning and high-ISA AVX/AVX2/AVX512/VNNI/FMA/F16C/BMI2 flags so the prebuilt binaries do not depend on the GitHub Actions runner CPU instruction set.

Use this version if v0.1.2 crashes with Illegal instruction, SIGILL, or Windows error 0xC000001D.

FunASR llama.cpp runtime v0.1.2

Prebuilt self-contained binaries for running SenseVoice (and Paraformer / Fun-ASR-Nano) locally with the FunASR llama.cpp / GGUF runtime — built-in FSMN-VAD, whisper.cpp-style on-device ASR, strong on Chinese & Cantonese.

New: q8 GGUF models are ~half the size of f16 with the same accuracy.

bash download-funasr-model.sh sensevoice ./gguf
llama-funasr-sensevoice -m ./gguf/sensevoice-small-q8.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav

No Python, no build. Linux (x64/arm64), macOS (arm64), Windows (x64). Docs: runtime/llama.cpp/README.md.

FunASR llama.cpp runtime v0.1.1

Prebuilt, self-contained binaries to run SenseVoice (and Paraformer / Fun-ASR-Nano) locally with the FunASR llama.cpp / GGUF runtime — built-in FSMN-VAD, whisper.cpp-style on-device ASR, strong on Chinese & Cantonese.

bash download-funasr-model.sh sensevoice ./gguf
llama-funasr-sensevoice -m ./gguf/SenseVoiceSmall-f16.gguf --vad ./gguf/fsmn-vad.gguf -a audio.wav

No Python, no build, no torch. Binaries for Linux (x64/arm64), macOS (arm64), Windows (x64). Docs: runtime/llama.cpp/README.md. CER (micro-avg): SenseVoice 8.01% vs whisper.cpp small 22%.

v1.0.0: SenseVoice — Multilingual Speech Understanding

SenseVoice v1.0.0

The first official release of the SenseVoiceSmall checkpoint for multilingual speech understanding.

Released checkpoint scope

SenseVoiceSmall supports ASR and language identification for Mandarin, Cantonese, English, Japanese, and Korean. It also returns emotion and audio-event tags. The broader SenseVoice research reports training on more than 400,000 hours and support for more than 50 languages; those research results should not be read as the released SenseVoiceSmall checkpoint's language list.

Highlights

• ASR and language identification for the five released languages.
• Speech emotion recognition with happy, sad, angry, and neutral labels.
• Audio event detection for events such as background music, applause, laughter, crying, coughing, and sneezing.
• A non-autoregressive architecture for low-latency inference. In the benchmark configuration documented in the repository, SenseVoiceSmall runs more than 5 times faster than Whisper-Small and 15 times faster than Whisper-Large at a similar parameter count.
• Optional composition with FunASR's separate FSMN-VAD and CAM++ models for segmented, speaker-attributed workflows. Speaker diarization is not a native SenseVoiceSmall checkpoint output.

Quick start

from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess

model = AutoModel(
    model="iic/SenseVoiceSmall",
    trust_remote_code=True,
    vad_model="fsmn-vad",
    vad_kwargs={"max_single_segment_time": 30000},
    device="cuda:0",
)
result = model.generate(
    input="audio.wav",
    cache={},
    language="auto",
    use_itn=True,
    batch_size_s=60,
    merge_vad=True,
)
print(rich_transcription_postprocess(result[0]["text"]))

Model

Model	Languages	Parameters	Download
SenseVoiceSmall	Mandarin, Cantonese, English, Japanese, Korean	234M	ModelScope / Hugging Face

Links

• Repository and benchmark details
• FunAudioLLM paper
• ModelScope demo
• Hugging Face Space
• FunASR toolkit