Add CTC HLG decoding for zipformer

[csukuangfj]

You can find a pre-trained zipformer CTC model and HLG.fst in the following repo
https://huggingface.co/csukuangfj/sherpa-onnx-zipformer-ctc-en-2023-10-02

HLG decoding

cd egs/librispeech/ASR

./zipformer/onnx_pretrained_ctc_HLG.py \
 --nn-model sherpa-onnx-zipformer-ctc-en-2023-10-02/model.onnx \
 --words sherpa-onnx-zipformer-ctc-en-2023-10-02/words.txt \
 --HLG ./sherpa-onnx-zipformer-ctc-en-2023-10-02/HLG.fst \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/0.wav  \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/1.wav \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/2.wav 

HL decoding

cd egs/librispeech/ASR

./zipformer/onnx_pretrained_ctc_HL.py \
 --nn-model sherpa-onnx-zipformer-ctc-en-2023-10-02/model.onnx \
 --words sherpa-onnx-zipformer-ctc-en-2023-10-02/words.txt \
 --HL ./sherpa-onnx-zipformer-ctc-en-2023-10-02/HL.fst \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/0.wav  \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/1.wav \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/2.wav 

H decoding

cd egs/librispeech/ASR

./zipformer/onnx_pretrained_ctc_H.py \
 --nn-model sherpa-onnx-zipformer-ctc-en-2023-10-02/model.onnx \
 --tokens sherpa-onnx-zipformer-ctc-en-2023-10-02/tokens.txt \
 --H ./sherpa-onnx-zipformer-ctc-en-2023-10-02/H.fst \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/0.wav  \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/1.wav \
 ./sherpa-onnx-zipformer-ctc-en-2023-10-02/test_wavs/2.wav 

[armusc]

Hi
I was trying the HLG decoding with a kaldi decoder, i.e. onnx_pretrained_ctc_HLG.py
the onnx model is fine because decoding without HLG gives good results
I built the HLG using convert-k2-to-openfst.py
I can see from printing the ilabels from the HLG best pah decoding that token ids are shifted by 1 with respect to decoded tokens from a HLG-less decoding (this is somehow explained in the comment # are shifted by 1 during graph construction)
the words on the olabel side are definitely not correct, they correspond to the "shifted by 1" tokenization
should I have done somethingdifferent in the HLG conversion?

thanks

[csukuangfj]

@armusc

Please have a look at

icefall/egs/librispeech/ASR/prepare.sh

Lines 307 to 309 in 7d56685

	./local/prepare_lang_fst.py \
	--lang-dir $lang_dir \
	--ngram-G ./data/lm/G_3_gram.fst.txt

You need to use ./local/prepare_lang_fst.py to generate H.fst, HL.fst and HLG.fst.

Any other scripts won't work, e.g., convert-k2-to-openfst.py will not work here.

Sorry for the confusion. We will update the documentation to describe how to use it.

[armusc]

thanks, it works

[armusc]

is HLG decoding with kaldi faster-decoder supposed to give the same results as k2-icefall 1best on the ctc output?
I observe a degradation, especially in terms of deletion, regardless of the beam value or max active;
in icefall 1 best, a lattice is first computed from which the 1best is extracted, I find beneficial setting the parameter hlg_scale (usually around 0.3 is optimal) and increasing this value leads to more deletion; (and btw, would it be possible to use the kaldi latgen-decoder on the ctc output to extract the 1 best from a kaldi lattice? would that make any difference?)

[csukuangfj]

and btw, would it be possible to use the kaldi latgen-decoder on the ctc output to extract the 1 best from a kaldi lattice

Yes, it is possible. I am working on it. Instead of 1best, you can get N-best, where N >=1.