Can I use another language without re-training?

[kikirizki]

Hi I wonder if this method can be used for any language, without retraining ? thank you

[Wendison]

Hi, that's a good question :) According to my testing, if the model is trained on one language, e.g., English, it can be also applied to Chinese but the performance is not as decent as English, e.g., the original content of Chinese utterances may not be well preserved. The key of successful conversion for unseen languages is that the VQCPC-based content encoder can accurately discover acoustic units related to underlying linguistic content or pronunciations of speech. Different languages may share some similar pronunciations, but there're still many different articulation units among different languages. Therefore, if the model is trained on multiple languages (i.e., letting the content encoder to discover more articulation units), it should have better generalization ability to unseen languages.

[kikirizki]

Thank you for your brief response and suggestion, I will try it