v1.0.0: SenseVoice — Multilingual Speech Understanding
SenseVoice v1.0.0
The first official release of the SenseVoiceSmall checkpoint for multilingual speech understanding.
Released checkpoint scope
SenseVoiceSmall supports ASR and language identification for Mandarin, Cantonese, English, Japanese, and Korean. It also returns emotion and audio-event tags. The broader SenseVoice research reports training on more than 400,000 hours and support for more than 50 languages; those research results should not be read as the released SenseVoiceSmall checkpoint's language list.
Highlights
- ASR and language identification for the five released languages.
- Speech emotion recognition with happy, sad, angry, and neutral labels.
- Audio event detection for events such as background music, applause, laughter, crying, coughing, and sneezing.
- A non-autoregressive architecture for low-latency inference. In the benchmark configuration documented in the repository, SenseVoiceSmall runs more than 5 times faster than Whisper-Small and 15 times faster than Whisper-Large at a similar parameter count.
- Optional composition with FunASR's separate FSMN-VAD and CAM++ models for segmented, speaker-attributed workflows. Speaker diarization is not a native SenseVoiceSmall checkpoint output.
Quick start
from funasr import AutoModel
from funasr.utils.postprocess_utils import rich_transcription_postprocess
model = AutoModel(
model="iic/SenseVoiceSmall",
trust_remote_code=True,
vad_model="fsmn-vad",
vad_kwargs={"max_single_segment_time": 30000},
device="cuda:0",
)
result = model.generate(
input="audio.wav",
cache={},
language="auto",
use_itn=True,
batch_size_s=60,
merge_vad=True,
)
print(rich_transcription_postprocess(result[0]["text"]))Model
| Model | Languages | Parameters | Download |
|---|---|---|---|
| SenseVoiceSmall | Mandarin, Cantonese, English, Japanese, Korean | 234M | ModelScope / Hugging Face |