about english cmu_lts_model.c and cmu_lex_data_raw.c

[zshy1205]

I have a question,can you help me?
why you cut the cmudict, and only 36964 english words in cmu_lex_data_raw.c
I know the cmudict contains 130000 english words, and I test the cmu_lts_model, it was performed poorly in cmu_lex_data_raw.c's 36964 words, about 90% word error rate. Why does this happen?(the cmu_lts_model is trained with cmudict which is removed the 36964 words? can you help me? thanks.
Forgive my poor English.

[fxwderrick]

@zshy1205 I found the same issue. Not sure why(maybe for smaller footprint). But you can extend/create larger cmulex dictionary following this blog https://boredomed.wordpress.com/2019/03/07/festvox-to-flite-tts-conversion/. There do exist a certain amount of pronunciation errors, even in the latest cmudict. And many words are not covered in the dictionary(that's why we need to model letter-to-sound for pronunciation guess). I suppose you could manually update the cmudict to fix them.

[zshy1205]

@fxwderrick Thanks, And I have one another question. There are some words have two or more Phoneme list, when you train the G2P model with cmudict, how to do with this case?

[fxwderrick]

@zshy1205 If you follow the steps in the blog, you will find the raw cmudict needs to be preprocessed first, like removing polyphone. Thus the input dict used for G2P training contains only one pronunciation for each word.

[zshy1205]

@fxwderrick thank u very much, I will read this blog carefully.

[bringtree]

@zshy1205 @fxwderrick I delete the polyphone. But the error rate is still high.

[bringtree]

I want to know how to use allowable and cmu_dict(0.4) to reproduce the good result. (the same as ./bin/t2p)

[bringtree]

@zshy1205 I check the error rate about CART. In the training data, the word error rate is about 60%. I also use sklearn package to reproduce this process in Python. The word error rate is about 60% in training data. So I think the word error rate is normal.
However, when I use some unseen words(Out of vocabulary in cmudict_0.4), the python version code' result is better than the flite(word error rate: 100%, letter error rate: 55%).
It is Unnormal.

[bfs18]

http://cmuflite.org/packed/flite-1.4/flite-1.4-release.tar.bz2 replacing tools/make_lts.scm with the corresponding file in the tarball before converting lts model to c format solves the problem.