Skip to content

FunASR llama.cpp runtime v0.2.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 11 Aug 04:53
· 4 commits to main since this release
runtime-llamacpp-v0.2.0
This tag was signed with the committer’s verified signature.
LauraGPT zhifu gao
SSH Key Fingerprint: AVarrACEcsMarEko5hsjhHKi7BlMDpzrwXPSuGcj/94
Verified
Learn about vigilant mode.
500956b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

FunASR llama.cpp runtime v0.2.0

Prebuilt, self-contained FunASR llama.cpp / GGUF binaries for Fun-ASR-Nano, SenseVoiceSmall, and Paraformer, with built-in FSMN-VAD. No Python ASR runtime or local C++ build is required.

Highlights

  • SRT subtitle output is available across Fun-ASR-Nano, SenseVoiceSmall, and Paraformer. Pass --srt to emit timestamped subtitle cues; the default plain-text output remains unchanged.
  • The bundled llama.cpp revision refreshes Vulkan submission batching for smaller AMD GPUs and improves VK_ERROR_DEVICE_LOST diagnostics.
  • AMD Vulkan troubleshooting now documents GGML_VK_MAX_NODES_PER_SUBMIT and GGML_VK_SERIALIZE_SUBMISSIONS, plus a CPU fallback. These changes improve diagnostics and batching behavior, but this release does not claim that every vendor-driver crash is fixed; please report the GPU model, driver version, command, and full log if a device still fails.

Download

Nine platform archives are attached:

  • Linux: linux-arm64, linux-x64, linux-x64-avx2, linux-x64-vulkan
  • macOS: macos-arm64
  • Windows: windows-x64, windows-x64-avx2, windows-x64-vulkan, windows-x64-cuda

Use the default x64 archive for maximum CPU compatibility. Choose x64-avx2 on CPUs with AVX2, FMA, F16C, and BMI2. The Vulkan archives require a working Vulkan driver/ICD and currently accelerate the SenseVoiceSmall graph with llama-funasr-sensevoice ... --backend vulkan. The Windows CUDA archive targets CUDA architecture 86 and requires a compatible NVIDIA driver; build from source for other GPU architectures.

Quick Start

Download the default quantized model with:

bash download-funasr-model.sh <sensevoice|paraformer|nano>

The helper requires the Hugging Face CLI (pip install -U huggingface_hub). Then run llama-funasr-cli, llama-funasr-sensevoice, or llama-funasr-paraformer.

Full documentation: https://github.com/modelscope/FunASR/blob/runtime-llamacpp-v0.2.0/runtime/llama.cpp/README.md

Assets 11
Loading