Skip to content

FunASR llama.cpp runtime v0.2.2

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Aug 15:29
· 167 commits to main since this release
runtime-llamacpp-v0.2.2
This tag was signed with the committer’s verified signature.
LauraGPT zhifu gao
SSH Key Fingerprint: JWOg46KsSROZJFYH9rn9stOEzW7fbotqgJ8PGwCZCXU
Verified
Learn about vigilant mode.
05be486
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer and Fun-ASR-Nano with built-in FSMN-VAD.

v0.2.2

  • Guards a failed CUDA or Vulkan ggml_backend_dev_init() before resolving the backend's default buffer type, so initialization failure is reported instead of dereferencing a null backend.
  • Flushes deterministic stderr boundaries before backend initialization, after initialization, and after buffer-type resolution. These messages distinguish selection, backend initialization, buffer resolution, and later graph/model failures.
  • Built from exact commit 05be4863fc94e3df37924a849b9790484ac74a29 with all nine platform jobs successful.
  • The Windows Vulkan package is ready for Radeon 780M and RX 9070 XT retesting. This release does not claim that the AMD Windows 0xC0000005 crash is fixed; a crash inside the Vulkan driver or backend initializer can still terminate before FunASR regains control.
  • Android/Mali remains outside the official prebuilt and continuously validated targets.

For --backend vulkan, interpret the final stderr boundary as follows:

  • No initializing vulkan backend ...: failure occurred during enumeration/selection or earlier.
  • initializing ... without initialized ...: failure occurred inside ggml_backend_dev_init() or below it.
  • initialized ...; resolving buffer type without vulkan backend ready ...: failure occurred while resolving the default buffer type.
  • vulkan backend ready ...: initialization completed and the failure is later in model loading or graph execution.

Quick start

Download the default quantized model with:

bash download-funasr-model.sh <sensevoice|paraformer|nano>

The helper requires the Hugging Face CLI (pip install -U huggingface_hub). Then run llama-funasr-cli, llama-funasr-sensevoice, or llama-funasr-paraformer.

Use the default x64 asset for maximum CPU compatibility. Use x64-avx2 on CPUs with AVX2/FMA/F16C/BMI2. The Vulkan assets require a working Vulkan driver/ICD and accelerate SenseVoiceSmall with --backend vulkan. The Windows CUDA asset targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

SHA-256

adc0e968d70a4308191a91011a3444e0a1cbefc90940c3e199b7d65ff9e59d0b  funasr-llamacpp-linux-arm64.tar.gz
cb5b5679938d2001426b5ea079ba948bac5c23b19aa2fc79e7a8572d9e9516e7  funasr-llamacpp-linux-x64-avx2.tar.gz
f865659d1787a2769d4ecfba598f2a490144819945bb2397fce3e172c1a1aff9  funasr-llamacpp-linux-x64-vulkan.tar.gz
c1ad11bac292288a783c1e5eb1103c6db301b98bf861d29b5b5310de3a190404  funasr-llamacpp-linux-x64.tar.gz
cb90c64c6c251d9df9a40193037713feaee9dd602d59b470bb4735d78c00da33  funasr-llamacpp-macos-arm64.tar.gz
f1c9ba8e35c273b995877e0fd7f4080df28e40bce2c41be96d862c44e20fea53  funasr-llamacpp-windows-x64-avx2.zip
52d5ecf4220e428737f9954b148d7ee1410109a12ef77fd45f99bc1c7dd040d3  funasr-llamacpp-windows-x64-cuda.zip
86a7d5ca7c134ae2fd3c9c1b356fcff041e29baf569c1145a75c73bb5bc5ea90  funasr-llamacpp-windows-x64-vulkan.zip
19e368fe0debaf880ae5aed063a1105e6cdc2d1a57f259b065df15db45a0103a  funasr-llamacpp-windows-x64.zip

Build evidence: https://github.com/modelscope/FunASR/actions/runs/33182316846

Docs: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.2/runtime/llama.cpp/README.md

Assets 11
Loading