Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix pause problem of Chinese speech #3351

Merged
merged 1 commit into from
Dec 4, 2023
Merged

fix pause problem of Chinese speech #3351

merged 1 commit into from
Dec 4, 2023

Conversation

aaron-lii
Copy link
Contributor

The pauses in Chinese speech generated by xtts pretrained model are somewhat unnatural. And I found that punctuation in Chinese text were replaced with spaces. I replaced punctuation with English comma, and it performs better.

Here's an example:

space

space.mp4

English comma

comma.mp4

Here's the text:
`> Using model: xtts

Text splitted to sentences.
['在汉字文化圈和海外华人社区中,中文也被称为汉文、华文。', '汉字主要起源于记事的象形性图画,象形字是汉字体系得以形成和发展的基础。', '后来的演变经历
了几千年的漫长历程,经历了甲骨文、金文、篆书、隶书、楷书、草书、行书等阶段,普遍使用楷书。']
Processing time: 53.86410617828369
Real-time factor: 1.6633757888038008`

I guess there are few punctuation in Chinese training data, so the pretrained model did not learn the meaning of spaces well. But it learned the meaning of English comma very well through English training data.

BTW, the model learned the meaning of spaces better when I fine-tuned it with my Chinese data with more punctuation.

@CLAassistant
Copy link

CLAassistant commented Dec 1, 2023

CLA assistant check
All committers have signed the CLA.

@erogol
Copy link
Member

erogol commented Dec 4, 2023

@aaron-lii nice catch and thanks for the PR !!

@erogol erogol merged commit e49c512 into coqui-ai:dev Dec 4, 2023
53 checks passed
@aaron-lii aaron-lii deleted the chinese-puncs branch December 7, 2023 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants