Zipformer for Common Voice

[yfyeung]

Data Preparation

The prepare.sh prepares the English (en) dataset of version 13.0 by default.

Version	Date	Size	Recorded Hours	Validated Hours	License	Number of Voices	Audio Format
Common Voice Corpus 13.0	3/15/2023	76.39 GB	3,209	2,429	CC-0	86,942	MP3

Stage	Time	Comment
Stage 0: Download data	-	Use machine in America to download the dataset manuallly
Stage 1: Prepare CommonVoice manifest	50 minutes	num_jobs=8
Stage 2: Prepare musan manifest	Very little	-
Stage 3: Preprocess CommonVoice manifest	5 minutes	-
Stage 4: Compute fbank for dev and test subsets of CommonVoice	2 minutes	1 GPU
Stage 5: Split train subset into 1000 pieces	10 minutes	-
Stage 6: Compute features for train subset of CommonVoice	3 hours	4 processes with 2 GPUs
Stage 7: Combine features for train	15 minutes	-
Stage 8: Compute fbank for musan	15 minutes	-
Stage 9: Prepare BPE based lang	5 minutes	-

Result

	Dev	Test
greedy search	9.96	12.54
modified beam search	9.86	12.48

To reproduce the above result, use the following commands for training:

export CUDA_VISIBLE_DEVICES="0,1,2,3"
./pruned_transducer_stateless7/train.py \
  --world-size 4 \
  --num-epochs 30 \
  --start-epoch 1 \
  --use-fp16 1 \
  --exp-dir pruned_transducer_stateless7/exp \
  --max-duration 550

and the following commands for decoding:

# greedy search
./pruned_transducer_stateless7/decode.py \
  --epoch 30 \
  --avg 5 \
  --decoding-method greedy_search \
  --exp-dir pruned_transducer_stateless7/exp \
  --bpe-model data/lang_bpe_500/bpe.model \
  --max-duration 600

# modified beam search
./pruned_transducer_stateless7/decode.py \
  --epoch 30 \
  --avg 5 \
  --decoding-method modified_beam_search \
  --beam-size 4 \
  --exp-dir pruned_transducer_stateless7/exp \
  --bpe-model data/lang_bpe_500/bpe.model \
  --max-duration 600

Pretrained model is available at
https://huggingface.co/yfyeung/icefall-asr-cv-corpus-13.0-2023-03-09-en-pruned-transducer-stateless7-2023-04-17

The tensorboard log for training is available at
https://tensorboard.dev/experiment/j4pJQty6RMOkMJtRySREKw/

[desh2608]

BTW it seems that num_jobs > 1 is currently not supported for the CommonVoice Lhotse recipe (see here). It may be worth implementing this option to speed up the manifest creation time.

[csukuangfj]

BTW it seems that num_jobs > 1 is currently not supported for the CommonVoice Lhotse recipe (see here). It may be worth implementing this option to speed up the manifest creation time.

There's an ongoing PR about that

lhotse-speech/lhotse#1025

[csukuangfj]

Could you please add a RESULTS.md to document the results, pre-trained models, tensorboard logs?

[yfyeung]

Sure

[csukuangfj]

Could you replace it with a symlink?

[csukuangfj]

Also for other files like joiner.py and model.py.

[yfyeung]

Sure.

[csukuangfj]

Thanks!