Voice navigation

[rinigus]

This PR adds voice navigation using an installed external TTS program. At present, mimic, flite, picotts, and espeak are supported. Description of this PR has been given in #30. However, for completeness, I am repeating it below. When compared to earlier description, support for OSRM and Mapquest Open was added.

Background

With the voice commands, we are having several restrictions. Namely, voice commands are not available in all languages and, it is probably common, user may want to specify the voice command language. Second, the voice synthesis availability is also rather poor. There are several options, more about it below. Third, when using better quality voices, synthesis may take some time. The developed code is able to handle these limitations, as much as I could.

Text to speech

After some search, I think we have three packages providing TTS in SFOS: mimic (based on flite), picotts, and espeak. Out of these, mimic supports English only, picotts has few other languages (de, es, fr, and it), and espeak has more. Quality of espeak is, though, rather poor. All packages are available at OpenRepos (mimic and picotts I have uploaded myself from OBS).

To handle these options, I made a class VoiceEngineBase that is later used as a base class by voice engines: VoiceEngineMimic, VoiceEngineFlite (supports flite if you prefer), VoiceEnginePicoTTS, and VoiceEngineEspeak. These voice engines are used by VoiceCommand that handles

• selection of the voice engine according to given language by searching an list of engines sorted by quality (subjective)
• getting requests for new voice commands (prompts) and asking voice engine to make it asynchronously
• giving a file with the voice corresponding to the command
• handling a cache of voice commands

Voice prompts

Voice prompts are given through Narrative and its Maneuver. Audio is handled through QML Audio by giving the audio file. When compared to earlier Maneuver, we have now to store and play verbal instructions. In Valhalla, its alert (like 200-300 meters before), pre-maneuver (turn left now), and post-maneuver (continue for 1 km). So, the routing engine (Valhalla and others) would have to provide these data. If only narrative is provided, its going to be used for alert and pre-maneuver prompt.

When compared to earlier Narrative, the proposed version has to define current_maneuver (to know when to play post-maneuver), interface with VoiceCommand, and remember which prompts have been voiced already (at present, the corresponding prompt is just deleted from a current maneuver copy after the prompts has been voiced). To keep the track of maneuvers, Narrative has begin and end methods called at the start and end of navigation by QML Map.

To support slower TTS (like mimic's ap voice), I made the voices to be synthesized in advance. At present, 3 maneuvers ahead. This should also allow us to extend to online synthesis, if we wish.

To ensure that we don't miss the maneuver, I reduced the period of narration timer to one second. At present, the preferred voice gender (when possible, only female voices in picotts) is specified in Preferences. Maybe we can move it to the new Navigation page.

Overall preference on whether to enable voice commands is stored in config.py/voice_commands, but doesn't have GUI, as you suggested.

[otsaloma]

Yes, I agree, the highest priority can be all kept.

[otsaloma]

Took a while, but I've now done the adaptation – mainly revised verbal prompts and overlap removal, using standard locales and fallbacks without hard-coded mappings and also small fixes and improvements such as saving maneuvers between sessions and rerouting voice prompts etc. Hope I didn't break anything. The commits are not very atomic, sorry about that, it's a result of file-by-file work and fixing all errors found in testing in one go.

I've done some desktop testing with the fake position source, it seems to work OK. I'll do more testing, but since I don't have a car, it'll be just desktop testing and actual field testing is appreciated to see if the parameters need tweaking. The most relevant parameters are:

• TTS speed

• Voice playback trigger threshold

• Verbal prompt offsets

And, @rinigus, do you actually use the voice navigation? When testing with Mimic, I found the pronounciation of Finnish road names to be horribly disturbing, at times it even seems to fall back on just enumerating the letters. It can't be much better in Estonian, or is it?

[rinigus]

Thank you very much! Its a huge change and I will read (and learn) the new code with pleasure. I hope you were not too frustrated with the original PR ...

As for use, yes, I do use it every time I am using navigation. A bit more in the summer than now, but still I do it every time I need to go out of the routine places. When you drive its of great help, even with these issues of streets names. I think picotts was doing a better job with the streets, but mimic has much better voice quality, so I made it the default. In addition, mimic/flite are developed in proper open source fashion, while picotts is non-developed anymore source code dump from Android.

State of TTS is far from ideal on Linux (and SFOS, as a result). However, by making apps using it we'll at least expose the problems and maybe someone (or us) get interest in fixing them.

The original PR was developed to have ability to plug in online TTS engines, if we wish. I'll have to see if this is still the case (whether we can pre-synthesise well in advance).

I am sad to see English Pirate go. I think it was very appropriate for Sailfish OS with all its "sailors". Its quite well developed by Valhalla and adds some personality to the navigation software. I wonder if its possible to restore it in Poor Maps?

I'll test the new version today and will report back.

[rinigus]

From testing: I think it works well. I tested it few times today (had to drive around a bit) and was prompted to make maneuvers in time. When I could, I checked the distance displayed on Poor Maps and compared it with the prompt text.

Few times, it seemed that the prompt was said 100 meters early, like at 400 meters I was told that its 300 to go. I'll read the code and look at the parameters that you listed earlier.

Couple of times it happened that post-maneuver text was interrupted by the next one. However, I didn't notice it on roundabouts which was an issue earlier. Although, more testing will be needed regarding it. (Please see the note below, though)

One question, before I forget: do we still have ability to have multiple prompts before maneuver? If the leg was really long.

Note that while I drive, I don't need perfect solution, but the one that helps. These prompts are very handy and I don't mind if they interrupt and tell the next, more relevant prompt. Its all to help the driver and keep the eyes on the road, not map, when we can. Right now, it already works well with prompts and occasional glancing over the map. :)

Off to read the new code...

[otsaloma]

Do you think it would make sense to add transformations for non-English letters? Using English TTS voice, the result for Finnish street names sounds much better at least with Mimic and Flite if I replace "ä" with "ae" etc. – try e.g. "Jämeräntaival" vs. "Jaemeraentaival"?

I am sad to see English Pirate go.

I understand, but I don't want to include something that hardly anyone uses more than once, if it takes up space on a small screen and needs special casing in code.

One question, before I forget: do we still have ability to have multiple prompts before maneuver? If the leg was really long.

Yes: https://github.com/rinigus/poor-maps/blob/voice/poor/narrative.py#L533

[rinigus]

Do you think it would make sense to add transformations for non-English letters? Using English TTS voice, the result for Finnish street names sounds much better at least with Mimic and Flite if I replace "ä" with "ae" etc. – try e.g. "Jämeräntaival" vs. "Jaemeraentaival"?

I would prefer to avoid it. In reality, its TTS issue which has to be solved properly by mimic - Valhalla interaction. Just imagine transliteration of some languages we are not aware about - I have no clue how it will work with Russian or Chinese, for example. I would prefer to feed it what we get from Valhalla and hope for the best.

Proper solutions in Linux are tricky - we either need someone to sell us a good TTS, use some online service, or maybe we can get Android TTS layer exposed in SFOS. I had written in a thread at TJC regarding state of TTS, but there was not much happening there. Maybe when navigation with spoken instructions will show up, things will become more interesting for everyone.

I understand, but I don't want to include something that hardly anyone uses more than once, if it takes up space on a small screen and needs special casing in code.

I am surely biased, but I actually used it more than regular English. Its somewhat amusing and surely brings you out of routine. No idea how much others were using it, though. [And here I went against my own policy of not fixing TTS by spelling out few words. But that was due to the fact that pirate-spoken TTS is a wayy too far from SFOS]. I do understand your reasoning as well.

Have to stop here and get some sleep. I'll continue tomorrow.

[otsaloma]

I would prefer to avoid it. In reality, its TTS issue which has to be solved properly by mimic - Valhalla interaction. Just imagine transliteration of some languages we are not aware about - I have no clue how it will work with Russian or Chinese, for example.

Yes, I figured there might be generality issues, and Wikipedia seems to agree, though Russian and Chinese surely don't use "ä"!

I'm still tempted to special case Finnish as there are a lot of users here, no TTS engine and those characters seem to trigger letter enumeration, which is quite bad, e.g. "taival" part of "Jämeräntaival". I'll give it some more thought.

locale = poor.util.get_default_locale()
if locale.startswith("fi") and self.language.startswith("en"):
    self.text = self.text.replace("ä", "ae")
    self.text = self.text.replace("ö", "oe")
    self.text = self.text.replace("å", "aa")

Edit 1: Default locale does not of course necessarily match the language of the street names, so it's far from perfect.

Edit 2: Or maybe I should just file a bug report against Mimic on this?

[otsaloma]

This is what I mean:

• Jämeräntaival: https://instaud.io/1xud
• Jaemeraentaival: https://instaud.io/1xue

[rinigus]

If we think it through, its quickly becoming unrealistic and maybe some hacky way is OK. Ideally

• Valhalla should convey the language info in the prompt. Something like
  <en>Turn right into</en><fi>Jämeräntaival</fi><en>street</en>. This is probably doable if we fill the issue in Valhalla and work on/test it. Street name language should be known by Valhalla and it should be possible to insert it into the resulting string, if requested. There is a standard that allows it do properly, see https://www.w3.org/TR/speech-synthesis11/#AppB

• Our multilingual TTS engine should be able to parse such string and generate the WAV file. And here it is quickly becoming a pipe dream. We have only one language TTS of decent quality (maybe add few more if we count picoTTS), but far from covering possible requirements. I think unless we get such engine that is able to process multilingual input, there is no big point in bugging Valhalla. However, it looks like mimic is expanding to support more languages. See repos available under https://github.com/MycroftAI?utf8=%E2%9C%93&q=mimic&type=&language= . I will have to look into it at some point and see whether I can release an updated version

Maybe, for now, we can just replace ä, ö and few other chars for any language (Estonian would benefit from it too) and see later how to progress from there.

[rinigus]

From reading the code - it looks nice and tidy, as expected, thank you very much! Reorganization and rewriting surely helped. I enjoyed reading it and I hope I didn't miss anything important.

Maybe we should cut a bit the distance at which we stop giving voice directions. Right now its 200 meters, maybe 100 meters are OK, as for rerouting. Right now, when I change the route, I sometimes get the instruction from the old route. This would reduce such event probability.

On my part, its ready :)

[otsaloma]

Maybe, for now, we can just replace ä, ö and few other chars for any language (Estonian would benefit from it too) and see later how to progress from there.

OK, I'm really not looking for more than small hack to work around the issue.

Maybe we should cut a bit the distance at which we stop giving voice directions. Right now its 200 meters, maybe 100 meters are OK, as for rerouting.

Sure. Thanks for testing, I'll make these remaining changes soon, then merge and do a couple other fixes before making a release.