Procedure

1. Data Preperation

hkust

1. File list (train|test/sph.flist)
2. Transcriptions
3. Collect
4. Normalization and Segmentation
5. Delete corrupted data
6. Make segment files from transcript (segments, sph.scp) segments file format is: utt-id side-id start-time end-time, e.g.: sw02001-A_000098-001156 sw02001-A 0.98 11.56
7. wav.scp (Map utt-id to file)
8. reco2file_and_channel (sw02001-A sw02001 A)
9. utt2spk, spk2utt

wsj

• text

  utt_id    WORD1 WORD2 WORD3 WORD4 ...
  

• segments

  utt_id    file_id    start_time    end_time
  

• wav.scp (may need convertion on the fly)

  file_id    path/file
  

• utt2spk

  utt_id    spkr
  

• spk2utt

  (can be generated with utils/fix_data_dir.sh data/train)

  spkr    utt_id1 utt_id2 utt_id3
  

2. Dictionay Preperation

hkust

1. Extract full vocabulary from generated 'text'

2. Split into English and Chinese

3. CMUdict

4. Strip stress and pronunciation variant markers from cmudict

5. Search for OOV words

6. Setup g2p and generate oov lexicon

7. Dowload pre-trained Sequitur G2P model

8. Prepare pronunciations for OOV words with the g2p model

9. cmu phoneme to pinyin phoneme

wsj

# [in data/local/dict/ ]
# lexicon.txt
# extra_questions.txt
# nonsilence_phones.txt
# optional_silence.txt
# silence_phones.txt 

1. Get CMUdict
2. Dict Prep
3. silence_phones.txt (literals, one per line)
4. nonsilence_phones.txt (each line is a list of phones that correspond really to the same base phone.)
5. extra_questions.txt (stress markings or tone markings)
6. lexicon.txt (the lexicon need to be reduced to only the words present in the corpus)
7. (optional, replaces lexicon.txt) lexiconp.txt (has the pronunciation probability as the second field)

3. Extract MFCC features

4. Train monophone models

5. Align audio with the acoustic models

6. Train triphone models

7. Re-align audio with the acoustic models & re-train triphone models

repeat the previous 2 steps with additional triphone training algorithms (delta+delta-delta training, LDA-MLLT, and SAT) for more refined models. The alignment algorithms include speaker independent alignments and FMLLR.