Recent Master snap erased Espeak Dictionary entries

[Brian1Gaff]

Updated a previous master this morning Tues 12 th sept 17, around 8am utc.
Basically its an installed copy on windows 7 pro64, and all the entries vanished after the update.
If this was a consequence of the Espeak update, then OK but if its something wrong with the update code users will not be pleased if an update removes the entries.
I was certainly not expecting it in a master branch update.

Edit:

Entries in the voice speech dictionary can be lost for espeak voices after updating NVDA.

See STR and explanation here: #7592 (comment)

[feerrenrut]

@Brian1Gaff When you say Espeak dictionaries, do you mean NVDA speech dictionaries (accessed via NVDA+n, preferences, speech dictionaries)?

Could you give us some more information:

• Do you have a log file that you can upload?
• What version were you updating from?
• How was the update triggered?

[leonardder]

I'm not trying toe be harsh here, but you seem to experience a lot of problems related to updating NVDA noone else seems to have, or at least, you are the only one who reports them. Would it be possible to create a clean installation of NVDA on your pc? I suggest coming back to this issue if you are able to reliably reproduce it, otherwise, let's just treat it as an incident. Again, just my opinion, and my opinion is not in anyway equivalent to whatever law.

[Brian1Gaff]

By definition, you cannot really have a clean install if it has to contain the dictionary entries, and as it never happened previously, and nothing has changed it does seem suspicious. Of course the only way to see is if it happens again. I'm experiencing several issues recently, this one the one where updating of some versions has errors or locks, and one for the word for in certain sentences which has only just started to occur. There is seemingly nothing wrong with the machine and as an example a real person would be leaving things to update, and not expect these issues either. I'll most certainly attempt to do a clean version over the weekend from an older, pre problem version as a portable as on Windows 7 at least, it is not so critical for usage to have an installed version. If that, seems to update and speak in a stable fashion then I will also do a complete erasure and reinstall of the working copy but that has a lot of custom settings and add ons I use for my charity work and I'd be loathe to throw it away, even though, in theory I could make it into a portable copy and simply reinstall it later if I needed to. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[Mary5958]

Could it be maybe the renaming of some eSpeak languages that have more forms, so for example what used to be only "English" before is now "English (Great Britain)"? So to get the dictionary working again you need to rename it accordingly or build a new one and move the contents of the old one to the new one.

[Brian1Gaff]

Erm I was not aware there were others for English. Could be. Might need to have a poke about then. Might also be worth mentioning in whats new if this is going to happen in a release version, so people are not going to be confused. bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field. -

[Brian1Gaff]

Thanks for this. The original file is still in the speech dics folder, but is apparently not being used by nvda any more. its obviously been moved somewhere, but thus far I cannot find it. This means, one hopes, that if I can find it and move/rename it to the new one, we would be back where we were before. If anyone on the team can tell me where to look, I'd be most grateful. I also see another dic in that file for the Microsoft speech platform and I've not tested if that works or not. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[Brian1Gaff]

OK here is the situation The bugs of updating and the dic entries seem to be one and the same. Try this on a master snap. Go to general settings and copy the current config to system, ignore the warnings about add ons and you will find it goes on forever, but an alt tab will show an error alert about cannot copy a file, and no matter how often one tries it says its in use or whatever. This never used to occur. Thus when any update is installed and a system update happens, the whole thing grinds to a halt with these errors and with a half run nvda, you seem as if you have crashed. If nobody else gets this problem then I'll have to investigate why I'm getting it. Maybe there is a corrupt file in the system files its copying to, who knows? I will have a poke about later on, but in the meantime can somebody tell me if it works for them? Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[Brian1Gaff]

Hi again. Going to the system files are and renaming the current new file name, the one with (united kingdom) in the name to something.old then renaming the old .dic file to the same name as the other one brought my dictionary items back as you suspected. And rather remarkably, also allowed the copy to system area button to execute successfully too. This probably means that the original file that I renamed was somehow corrupted. My only question now is that should I rename the file not in the systems are to the same as the (united kingdom) one or should that have happened when the update occurred? If it was supposed to it failed. Now to look at some issues on the portable copies. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[Mary5958]

I would probably rename also the one in the speechDicts folder so that you would have it independently on the system area; BTW, it worked for me as I tested it here. I don't think the files would have renamed automatically in the update so whoever uses languages that have more forms will most likely experience the same. Anyway, nice that you got it working again!

[leonardder]

The voice renames are probably related to #7385.

[Brian1Gaff]

As you will see, most of the problems experienced recently were due to the update not actually copying and renaming the .dic file to the one with the (united kingdom) in possibly due to the file resisting being overwritten by the system. this broke the up date and hence allowed incorrect data about all sorts of bits of nvda to exist in the two main places in the system and program files folders. Finally managing to rename the problem file has meant that the next update to master actually fixed all the underlying issues of crashing incorrect names and bits of the dictionarries going missing. the latter were still there but named wrongly. It also seemed to result in add ons lists being different in system and program settings which was why the core issue was created in another ticket. Off to sort out the portable copies, which should be a lot easier as there is only one place the files are stored. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[Brian1Gaff]

No it renamed the files properly once the suspect one was finally renamed in windows. It also has properly fixed the rest of the config as well as some add ons were out of sync, which no doubt caused the crash on reboot and sundry other inconsistancies. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[feerrenrut]

I'm sorry, I'm having a hard time following this thread, I have tested this out and can reproduce the issue.

1. Do a clean install of 2017.3 (first delete %appdata%/nvda)
2. Ensure the synthesizer is set to espeak (NVDA settings, preferences, synthesizer)
3. Ensure that the voice is set to english (NVDA settings, preferences, voice settings)
4. Open settings, preferences, speech dictionaries, voice dictionary
5. Add an obvious entry that you can look for later EG "NEW ENTRY IN 2017.3"
6. Install latest master (I installed nvda_snapshot_master-14436,b73a9583)
7. Open speech dictionaries, notice that "NEW ENTRY IN 2017.3" is missing.

Note

• The file is still in %appdata%/nvda/speechDicts as espeak-english.dic
• Looking at the following snippet from def loadVoiceDict in source/speechDictHandler.py, its clear that the name of the voice is being used rather than the ID of the voice.

voiceName = synth.availableVoices[synth.voice].name
fileName=r"%s\%s-%s.dic"%(speechDictsPath,synth.name,api.filterFileName(voiceName))

When fixing

There are two parts to fixing this, recover the "lost" espeak voice dict files, and improving the robustness of the voice dict files overall.

1. To improve the robustness, and to be consistent with how we refer to a voice for a synth in the config file, we should move to use the ID of the voice instead of the voice name.
   • This will require a step to rename any existing files to the new format.
   • Consider that we can not load all synthesizers to get their IDs, some synthesizers may show a license agreement or similar. If this is not triggered by a user action it could be quite problematic for the user.
   • We need a reliable (and computationally cheap) way of determining if any files need to be updated for the current synth.
2. To recover the espeak voice dict files we will need to manually create a mapping for each of the names of voices (languages, not variants) to their voice ID (as given by the synthesizer).
   • Because this is a known transformation, we do not need the espeak synth to be loaded to do it.

[feerrenrut]

Setting priority to P1 since this has the potential to affect a lot of users, and fixing before the 2017.4 release is likely to prevent a lot of clean up work.

Work around

For those experiencing this issue on next or master builds, You may take the following steps, but this may mean that the future fix will not apply correctly and you will have to manually fix the dictionary names. Alternatively you can revert to 2017.3 until this issue is fixed.

1. Open %appdata%/nvda/speechDicts (perhaps first make a backup of this whole folder in case you make a mistake, just restore the folder)
2. Move all espeak-*.dic files to a new location, for instance a 'espeakDicts` directory on your desktop.
3. For each espeak-languageName.dic file in your desktop/espeakDicts/ folder (where languageName is an espeak voice name) EG: espeak-english.dic
   1. Open the nvda settings, preferences, voice settings dialog
   2. Set the most similar voice name and press ok., for instance in the case of espeak-english.dic, English (Great Britain)
   3. Open nvda settings, preferences, speech dictionaries, voice dictionary.
   4. Add an entry, EG test
   5. In %appdata%/nvda/speechDicts take note of the new espeak-languageName.dic file.
   6. Rename the equivalent file in your desktop/espeakDicts/ folder to match the new file in the previous step.
4. Copy each file from desktop/espeakDicts/ into %appdata%/nvda/speechDicts overwriting the files there.

[Brian1Gaff]

Which is basically what I did after killing the file which got corrupted somehow of course. I've not had a failed update since this. Maybe it might be an idea to push out a version of.3 with a renamer etc in it or at least some way to copy contents across, then all future updates would work. The point means that perhaps very few people have set up dictionary entries in their voice dics. I'd imagine a small batch process could do the renaming if it were just simply one language but if you wanted to do this for the local language each time and possibly others my brain would struggle to write such a file I think. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field.

[Brian1Gaff]

PS I did my renaming while running a portable version so no issues with in use files etc. Sorry if my long thread investigating this was confusing. Not intended. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field. ----- Original Message ----- From: "Reef Turner" <notifications@github.com> To: "nvaccess/nvda" <nvda@noreply.github.com> Cc: "Brian Gaff" <bglists@blueyonder.co.uk>; "Mention" <mention@noreply.github.com> Sent: Monday, September 18, 2017 8:24 AM Subject: Re: [nvaccess/nvda] Recent Master snap erased Espeak Dictionary entries (#7592)

…

Setting priority to `P1` since this has the potential to affect a lot of users, and fixing before the 2017.4 release is likely to prevent a lot of clean up work. ### Work around For those experiencing this issue on `next` or `master` builds, You may take the following steps, but this may mean that the future fix will not apply correctly and you will have to manually fix the dictionary names. Alternatively you can revert to 2017.3 until this issue is fixed. 1. Open `%appdata%/nvda/speechDicts` (perhaps first make a backup of this whole folder in case you make a mistake, just restore the folder) 1. Move all `espeak-*.dic` files to a new location, for instance a 'espeakDicts` directory on your desktop. 1. For each `espeak-languageName.dic` file in your `desktop/espeakDicts/` folder (where `languageName` is an espeak voice name) EG: `espeak-english.dic` 1. Open the nvda settings, preferences, voice settings dialog 1. Set the most similar voice name and press ok., for instance in the case of `espeak-english.dic`, `English (Great Britain)` 1. Open nvda settings, preferences, speech dictionaries, voice dictionary. 1. Add an entry, EG `test` 1. In `%appdata%/nvda/speechDicts` take note of the new `espeak-languageName.dic` file. 1. Rename the equivalent file in your `desktop/espeakDicts/` folder to match the new file in the previous step. 1. Copy each file from `desktop/espeakDicts/` into `%appdata%/nvda/speechDicts` overwriting the files there. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #7592 (comment)

[PratikP1]

I encountered a similar issues when testing the OneCore voices during the 2017-3 development. I reported it in the OneCore thread but didn't escalate it as a bug. At some point in the development cycle for Windows 10 Fall Creators update, the voice names for OneCore voices changed and the dictionary files had to e renamed in order to reflect these changes.

[Brian1Gaff]

That should not have affected Espeak though, surely? Besides the issue seems to solve itself in spek mostly if you remove the old file that seems to get mangled. Not running 10 here so cannot comment about that synth. I do have synths in sapi 4 5 speech platform and pico though and they still seem to be working but cannot recall if the dictionaries were the same or not. Brian bglists@blueyonder.co.uk Sent via blueyonder. Please address personal email to:- briang1@blueyonder.co.uk, putting 'Brian Gaff' in the display name field. ----- O