Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Letter-by-letter pronunciation not possible #2619

Closed
eginhard opened this issue May 16, 2023 · 2 comments · Fixed by #3286
Closed

[Bug] Letter-by-letter pronunciation not possible #2619

eginhard opened this issue May 16, 2023 · 2 comments · Fixed by #3286
Labels
bug Something isn't working

Comments

@eginhard
Copy link
Contributor

Describe the bug

I'd like to force the TTS model to pronounce a word letter by letter, e.g. "ARD" should be pronounced "A R D" (/ˌeɪˌɑːɹdˈiː/). In systems with SSML support (#752) you could use <speak><say-as interpret-as="verbatim">ard</say-as></speak>, but another way would be fine as well.

Espeak supports this even for words not in its dictionary by adding periods between the characters: espeak-ng --ipa -v en-us "A.R.D." is read /ˌeɪˌɑːɹdˈiː/.

This doesn't work in Coqui because the input for Espeak is split at punctuation characters and each chunk ["A", "R", "D"] is phonemized separately:

text, punctuations = self._phonemize_preprocess(text)

This results in the word, not the letter pronunciation of "a" being chosen (ɐ instead of eɪ). I could change _phonemize_preprocess() to pass the input to Espeak with punctuation included, but I'm not sure about the side effects. Is there a specific reason to do it this way?

To Reproduce

from TTS.api import TTS
p = TTS(model_name="tts_models/en/ljspeech/vits", gpu=False).synthesizer.tts_model.tokenizer.phonemizer
p.phonemize("A.R.D.")

Output: 'ˈɐ.ˈɑːɹ.d|ˈiː.'

Expected behavior

Expected output: ˌeɪˌɑːɹdˈiː

Logs

No response

Environment

{
    "CUDA": {
        "GPU": [],
        "available": false,
        "version": "11.7"
    },
    "Packages": {
        "PyTorch_debug": false,
        "PyTorch_version": "2.0.0+cu117",
        "TTS": "0.10.2",
        "numpy": "1.22.4"
    },
    "System": {
        "OS": "Linux",
        "architecture": [
            "64bit",
            "ELF"
        ],
        "processor": "x86_64",
        "python": "3.10.8",
        "version": "#42~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 18 17:40:00 UTC 2"
    }
}

Additional context

No response

@eginhard eginhard added the bug Something isn't working label May 16, 2023
@erogol
Copy link
Member

erogol commented May 16, 2023

It is possible but needs a model that is capable of doing that. I close this since it is not a dev issue.

@erogol erogol closed this as completed May 16, 2023
@eginhard
Copy link
Contributor Author

The output is fine with the tts_models/en/ljspeech/vits model. E.g. "ABC CNN ESPN" is correctly read letter by letter because all words are marked as abbreviations in the Espeak dictionary.

Espeak doesn't know about other letter sequences, like "ARD" or "ABCDEFGHIJKLMNOPQRSTUVWXYZ", and tries to read them as a word. I can force it to phonemize them as letter sequences by adding periods between each letter, but Coqui strips all punctuation before calling Espeak. Changing

return self._punctuator.strip_to_restore(text)
to return [text], [] fixes this and results in correct letter-by-letter output for "A.R.D" and "A.B.C.D.E.F.G.H.I.J.K.L.M.N.O.P.Q.R.S.T.U.V.W.X.Y.Z". But I'm not sure if there is a specific reason that Coqui strips the punctuation there and changing it wouldn't cause other issues?

eginhard added a commit to idiap/coqui-ai-TTS that referenced this issue Nov 22, 2023
Previously, the text was wrapped in an additional set of quotes that was passed
to Espeak. This could result in different phonemization in certain edges and
caused the insertion of an initial separator "_" that had to be removed.
Compare:
$ espeak-ng -q -b 1 -v en-us --ipa=1 '"A"'
_ˈɐ
$ espeak-ng -q -b 1 -v en-us --ipa=1 'A'
ˈeɪ

Fixes coqui-ai#2619
erogol pushed a commit that referenced this issue Nov 24, 2023
Previously, the text was wrapped in an additional set of quotes that was passed
to Espeak. This could result in different phonemization in certain edges and
caused the insertion of an initial separator "_" that had to be removed.
Compare:
$ espeak-ng -q -b 1 -v en-us --ipa=1 '"A"'
_ˈɐ
$ espeak-ng -q -b 1 -v en-us --ipa=1 'A'
ˈeɪ

Fixes #2619
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants