Add MMI training with word pieces as modelling unit.

[csukuangfj]

Will post the results once they are available.

[pzelasko]

IIRC the alignments from alimdl did not help before when we checked them in snowfall; do you expect different results with the current setup?

[danpovey]

We are thinking they might help training get started, where that's a problem, e.g. for MMI with BPE.

[csukuangfj]

I just started the MMI training with pre-computed alignments.

The tensorboard logs are:

• without attention decoder: https://tensorboard.dev/experiment/RsOln3DOSROOTQw9qDvcGA/#scalars&_smoothingWeight=0

• with attention decoder: https://tensorboard.dev/experiment/J00TO00nQpSyzV6tgc6u2Q/

---

Without attention decoder

It throws the following warnings at some point (after several hundred batches):

At some other point, it stops printing the above warnings and the MMI loss starts to decrease:

---

You can see that pre-computed alignment is helpful to make the training converge.
(We will see whether it will diverge later)

[csukuangfj]

The best WER I get for this pull request is

Training without attention decoder

(decoding using whole-lattice-rescoring, i.e., HLG 1-best decoding + 4-gram LM rescoring)

• test-clean: 2.79
• test-other: 6.39

Training attention decoder

(decoding using attention decoder for rescoring)

• test-clean: 2.82
• test-other: 6.67

---

LF-MMI + attention decoder seems not as good as CTC + attention decoder.
I will do more experiments on it after finishing the decoding script for #54

---

Let's merge it first since it contains code for integrating framewise alignment information into training, which can
be used by @danpovey

[pzelasko]

Are there any change in this file? Was it supposed to be a symlink like the other asr_datamodule.py in conformer_ctc?

[csukuangfj]

It is the same as the one in conformer_ctc and tdnn_lstm. I should have placed a symlink here.

[pzelasko]

Why is it named UniqLexicon? Not sure how to interpret it.

[csukuangfj]

Uniq here means each word in the lexicon has only one pronunciation, i.e., a unique pronunciation.

In BPE based lexicons, each word can be decomposed in a deterministic way.
In phone based lexicons, if a word has more than one pronunciation, there are scripts to keep only the first one.

[pzelasko]

Is this intended to be commented out?

[csukuangfj]

Yes, the non_optimized version is easier to understand and consumes less memory.