This is a phoneme ASR model based on the whisper-small model. The features are extracted using the Whisper encoder output and training is carried out using the MFA tool to obtain alignments between phonemes and frame-level Whisper features.
The dictionary used is opencpop-strict, and the training is carried out using a portion of the Aishell3 dataset.
Advantages
- Can recognize phonemes as well as tones
- Can directly recognize the phoneme and the duration
- Good performance on data with clear pronunciation
Disadvantages
- Only supports Chinese language
- Limited generalization performance due to small training dataset, and poor performance on data with unclear pronunciation
- May recognize some illegal phoneme sequences, such as "d ang ong"
- （别抱太高期望，很多时候会识别的一坨屎）
Download recognition_model.pth and place it in the assets directory. Place the audio files in the dataset_raw directory by speaker and execute batch_annotate.py.
To recognize a single audio file, use infer.py.