Add optional unicode normalization before passing strings to speech or braille

[LeonarddeR]

Is your feature request related to a problem? Please describe.

In some cases, text can contain ligature characters that are not provided in a braille table. Alternatively, a speech synthesizer kan really struggle with these.

An example is the ligature ij (ij), in dutch as in ijsbeer (polar bear). The Dutch version of ESPeak is unable to pronounce this word correctly.

An exactly opposite example is á, which is composed of two characters, namely the letter a and the modifier ́.

Describe the solution you'd like

For both speech and braille, i propose adding the ability to enable unicode normalization with the NFKC algoritm (Normalization Form Compatibility Composition). This algorithm ensures that most ligatures are properly decomposed before passing them to the synthesizer while composing characters like á, which is much more common than á.

Note that while this sounds utterly complex, it is basically adding one line of code:
processed = unicodedata.normalize("NFKC", original)

[CyrilleB79]

Some answers/reactions:

1. Unicode normalization on "bœr" does not seem to change anything on my end. How does it improve things for Dutch reading?
2. Is the issue only with eSpeak or with many TTS?
3. I have not noticed any issue with "œ" character in French TTS. This ligature is also used in French but TTS seem to behave the same no matter if we have the ligature or two separate characters ("oe")
4. Regarding the characters with diacritic, using one compound character ("á") behaves much better with TTSs then two characters ("á") at least when reading French.
5. I think that such normalization is offered and customizable in at least one TTS; I do not remember which one.
6. I do not know the difference between NFKC and NFC; is one normalization form preferable and why?

[LeonarddeR]

1. Unicode normalization on "bœr" does not seem to change anything on my end. How does it improve things for Dutch reading?

I'm very sorry. I actually performed tests for ij and somehow assumed it also worked for œ, but turns out it doesn't. For ij it definitely works: unicodedata.normalize("NFKC", "ijsbeer") returns ijsbeer
I updated the initial description accordingly.

2. Is the issue only with eSpeak or with many TTS?

I can reproduce the issue with ijsbeer with ESPeak, OneCore and Vocalizer Expressive.

3. I have not noticed any issue with "œ" character in French TTS. This ligature is also used in French but TTS seem to behave the same no matter if we have the ligature or two separate characters ("oe")

For the Dutch ESpeak, bœr pronounces as bor (i.e. it behaves like only the o is present). Vocalizer and OneCore seem to ignore the ligature completely.

4. Regarding the characters with diacritic, using one compound character ("á") behaves much better with TTSs then two characters ("á") at least when reading French.

Same applies to Dutch.

6. I do not know the difference between NFKC and NFC; is one normalization form preferable and why?

The NFC variant doesn't provide compatible characters for ligatures and therefore doesn't touch them. While NFKD does decompose the ligeature, it also decomposes á. Therefore NFKC seems to be the only method that really makes sense.

That said, I really want to know why œ is left alone and ij is decomposed correctly.

[dkager]

This sounds like a good idea. Would it be possible to normalize only once if the user had both speech and braille enabled? While the overhead seems small to me, it wouldn't hurt to try and minimize it.

[LeonarddeR]

I think that even when we normalize on the TextInfo level, normalization would happen once for braille and once for speech.
That said, I now realize that normalizing might break offset based text infos. Need to check that.

[thgcode]

A question, does the normalization could solve the problem of italic or bold unicode characters that synthesizers can't read?

[LeonarddeR]

Do you have examples of this? I'm certainly willing to investigate. In the end, I'm searching for a normalization strategy that works best for anyone.

[ABuffEr]

Do you have examples of this? I'm certainly willing to investigate.

Hi @LeonarddeR,
this should be a good resource. If you want, see also this add-on, that uses Unidecode library.

[LeonarddeR]

Yes, 𝒊𝒕𝒂𝒍𝒊𝒄𝒔 is normalized to italics

[ABuffEr]

Thus, a such PR is fundamental, in my opinion ⚡
Seriously, whole keywords, titles and phrases are written with these characters nowadays, on posts of social networks, nicknames in chats, and so on. In the majority of cases, NVDA simply ignores them.
Working to fix textInfos offsets should bring to a Braille emoji translations too, that is a welcome feature directly in core, for me (and other users with hearing problems, I think).
And I appreciate a lot even ligature fix, that struggled me from time to time during university studies.

[LeonarddeR]

Actually, thanks to very valuable work by @mltony in #16219, I think we can make this work.
We can create a new offset converter for normalization and use that to map normalized positions to real raw positions in the text. This way cursor routing and presentation should still work.

[LeonarddeR]

I created an offset converter that seems to do this reliably now. We can add this as an optional feature to speech and braille output.

[Adriani90]

Why should this be optional?

[CyrilleB79]

It should be configurable because there are both uses cases with the feature on and off:

• normalization off (as today): allows to detect easily letter-like characters used in fishing e-mails, either because your synth ignores it or because it reports it differently
• normalization on:
  • useful to be able to read with some TTSs letters with diacritics written with two separate characters instead of a combined one.
  • useful to read text written with specific unicode characters (e.g. unicode italic or unicode bold) instead of standard ones, e.g. frequently used for nicknames in some forums