Skip to content

FunASR llama.cpp runtime v0.2.4

Choose a tag to compare

View all tags
@github-actions github-actions released this 29 Aug 04:00
· 23 commits to main since this release
runtime-llamacpp-v0.2.4
This tag was signed with the committer’s verified signature.
LauraGPT zhifu gao
SSH Key Fingerprint: JWOg46KsSROZJFYH9rn9stOEzW7fbotqgJ8PGwCZCXU
Verified
Learn about vigilant mode.
c8d43b1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Prebuilt self-contained binaries for the FunASR llama.cpp / GGUF runtime: SenseVoice, Paraformer, and Fun-ASR-Nano with built-in FSMN-VAD.

v0.2.4

  • Fixes intermittent empty SenseVoiceSmall transcripts when using F16 GGUF model weights.
  • Decodes query embeddings according to their GGML F16/F32 tensor type, validates the embedding width, and rejects out-of-range token IDs instead of reading F16 storage through a float *.
  • The exact v0.2.3 Linux AVX2 asset reproduced 22 blank outputs across 298 completed runs. The fixed source produced one identical non-empty result in 100/100 runs and matched the Q8 model output byte-for-byte.
  • Built from exact tag target c8d43b10dedcb41f984f96e934a7fc63c9a6de3f; all nine platform builds and the release job succeeded. The source fix landed in #3550.

This release fixes the F16 embedding-read defect on the supported runtime paths. It does not claim to fix the separate Windows AMD Vulkan 0xC0000005 driver/backend crash tracked in #3479. Android/Mali remains outside the official prebuilt and continuously validated targets.

Quick start

Download the default quantized model with:

bash download-funasr-model.sh <sensevoice|paraformer|nano>

The helper requires the Hugging Face CLI (pip install -U huggingface_hub). Then run llama-funasr-cli, llama-funasr-sensevoice, or llama-funasr-paraformer.

Use the default x64 asset for maximum CPU compatibility. Use x64-avx2 on CPUs with AVX2/FMA/F16C/BMI2. Vulkan assets require a working Vulkan driver/ICD. The Windows CUDA asset targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

SHA-256

acdda3aae906ab77ae99852336d4989a863e4125182ffc74ec6523aa6f9c8353  funasr-llamacpp-linux-arm64.tar.gz
fec98150cd2fe845df0150e950faf17319a46b67a8c621ad24c659ba26a1ab72  funasr-llamacpp-linux-x64-avx2.tar.gz
169f8a6532a31d9c4a9bf6ecfe5db9949f0bf78d362dc6791ef6ac4ee42faa48  funasr-llamacpp-linux-x64-vulkan.tar.gz
51951c8d916b4babd873780b2070b03351cf0e7fda572955c81c120439158d7e  funasr-llamacpp-linux-x64.tar.gz
d58dda9da783733d59797bbec49568cf3984c3b783ab152f039b22c24d3ac10e  funasr-llamacpp-macos-arm64.tar.gz
97b4783ec5366be637621afad213ed4f3427bee1ff49e2d4f1561535fe96842c  funasr-llamacpp-windows-x64-avx2.zip
6c4b1bbc68a27d6acf5588e3195b85229d128ec5f576d8e47ba71611a206e1f1  funasr-llamacpp-windows-x64-cuda.zip
a1b46b31c31b32cac2e91e07b5718afb9289f47f9d22122da8ab20f630d37276  funasr-llamacpp-windows-x64-vulkan.zip
7beb08c3d7376643b69a4b280ea85a426e8d7cbef694290e5aa3c8d3589c75cd  funasr-llamacpp-windows-x64.zip

Build evidence: https://github.com/modelscope/FunASR/actions/runs/33231441888

Fix: #3550

Docs: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.4/runtime/llama.cpp/README.md

Assets 11
Loading