NVDA add automatically speak character description function when cursor move

[LittleStar-VIP]

Is your feature request related to a problem? Please describe.

As I am a non-English user, I have to rely heavily on the character description function of NVDA, i.e. pressing numpad 2 twice to report the character description of character at current cursor focus.
Please imagine that, typing a piece of article and you have to press numpad 2 twice in almost every word is a nightmare.
In fact, I have been using the addon enhance Phonetic reading for years, but I have contacted the addon author several times, hope that the author can make the addon compatible with latest nvda version, but I have not received any feedback from the author.
As this function is really key to my daily usage, I hope nvda can one day add this function to the core application.

Describe the solution you'd like

1. when user moves the cursor, both edit or review cursor, e.g. pressing right arrow
2. after certain delay, e.g. after user defined number of seconds, nvda report character description of the character in focus automatically
   e.g. for English character, pressing arrow key may cause nvda to report a alpha automatically
   e.g. for non English character that has more than one character description, like Chinese, nvda can add a user defined variable to set the maximum number of character description to be automatically spoken

Describe alternatives you've considered

Additional context

[bdorer]

I like this idea. This would also help persons with hearing dificulties.

[XLTechie]

Agreed. This function has been in iOS's VoiceOver as standard practice for a decade, and it is not something I've heard any complaints about. It can be disabled there, but is enabled by default. I too would like to see this one in core @feerrenrut. I suspect it would not impose much of a maintenance burden, at least from a past review I did of the add-on code.

[Brian1Gaff]

Yes it would be very handy for some voices where it tends to sound very similar on different characters, I'd imagine.

…

I like this idea. This would also help persons with hearing difficulties.

[cary-rowen]

Yes, this feature is also useful in Chinese environment. Hope to see it in NVDA core.

[feerrenrut]

Yes, we are happy for someone to start work on this.

[davidacm]

Hi. I'm the developer of the add-on, I could implement it on the Core of NVDA. Although I have some doubts about the implementation. E.G. Python threads or wx.CallLater for the timers? Initially I used python threads, but that way had some incompatibilities with other add-ons. In the last beta version of this add-on, I've used wx.CallLater, and this workaround fixed those issues. But I don't know if this is correct, or can introduce another issues, although no one has reported issues so far.

[davidacm]

Hi @LittleStar-VIP, sorry for my delaied response hehe. I usually receive a lot of e-mails and sometimes some messages go to the spam, etc.

The enhanced phonetic reading add-on, is compatible with the last stable version of NVDA. You can test the latest beta version from this link:
Enhanced phonetic reading

This version fix some issues, for example incompatibilities with bluetooth audio add-on. But could introduce new issues, because I used wx.CallLater and not python threads as in older versions.

I going to work in an implementation of this feature on the NVDA core, but in the meantime, you can use this version and you and others can send me feedback on the GitHub repo

[davidacm]

Hi everybody. What do you think would be a good category to place the character descriptions settings?
is the way NVDA would handle character reading... But I can't figure out what would be a good category for this. For testing purposes maybe Advanced options initially I guess, although isn't a dangerous feature.

[cary-rowen]

Hi @davidacm

Glad you'd like to implement this feature in NVDA core, I tend to place this option in NVDA's voice settings panel, This feature is somewhat similar to pitch change.

Thanks

[XLTechie]

What do you think would be a good category to place the character descriptions settings?

I think Speech comes the closest to being the correct panel. Less desirable options are General, Keyboard, and Object Presentation, all of which seem inappropriate in one way or another, though better than Advanced. Still, I think Speech settings should work.

[LittleStar-VIP]

@davidacm Thanks very much, this addon really help me a lot.

[LittleStar-VIP]

Yes, probably speech setting is the best location, unless new verbosity category is added.

[XLTechie]

@davidacm I think wx.CallLater would work, but you might find that core.callLater works more solidly, with less chance of possible problems. Although I am unclear what problems may arise with this particular usage. Perhaps @feerrenrut or @seanbudd know which would be preferable, or if some other direction would be better? But absent comment from one of the core devs, I would suggest core.callLater.

[feerrenrut]

Hard to give advice on whether core.callLater will fix the problems because I don't know what they were.

I'll try to give some background information instead.

wx.CallLater is typically used for GUI related behavior. Often code needs to respond to some event, but it wouldn't be correct to make a change while in the middle of processing that event. In these cases you defer the handling with wx.CallLater. It finishes processing the event and executes the call afterwards.

core.callLater is really just a wrapper around wx.CallLater, but it ensures that wx.CallLater is called from the main thread.

queueHandler.queueFunction(queueHandler.eventQueue, ... is a different but related approach. It does not allow for deferring by a set time, and there is no guarantee the call will happen immediately. It happens at the next core event pump. This is commonly used to execute UI messages. This module really needs better documentation, I'm concerned that it's usage doesn't follow the original intent.

[XLTechie]

Hard to give advice on whether core.callLater will fix the problems because I don't know what they were.

@feerrenrut I'll ask the question differently then, since I don't know what the problems with threads were either. After the reviewed character is spoken, in order to speak the phonetic after the (user-configured) delay, he needs to be able to perform an interruptable speech action, after a configured number of milliseconds. How would you do that?

[feerrenrut]

Probably core.callLater is the approach you want, it allows for delays and handles running on the main thread.

UI.message will report (with speech and braille) a message. Speech can be interrupted by keyboard events (pressing any key), assuming that is what you mean by interruptable.

Why the (user-configured` delay? Can you give a solid example of the expected speech:
It sounds like you mean.

- User presses numpad 3 to move the review cursor by character
- NVDA reports the character `a`
- After a delay of X NVDA reports `alpha`

Why not report the character description immediately after the character?
E.G.

- User presses numpad 3 to move the review cursor by character
- NVDA reports the character `a  alpha`

[LittleStar-VIP]

Why not report the character description immediately after the character?

As a user, I would like to speak on this issue.

1. As this automatically speaking of character description doesn't apply just to English character, giving a delay allow user to clearly identify the character from the character description

2. Also, I need not hear the character description of every character, e.g. a and c you will hardly mess up the two, but for p and b, m and n you may have the possibility of missing out the character at the first glance, so, if I want to hear the character description I will stop and wait for some seconds, if otherwise, it may be speaking too much info for each character.

[derekriemer]

I implemented this and I did it with speech commands, specifically the BreakCommand. There are two clear advantages to this:

1. Speech functionality doesn't have to do any funky callLater or anything, it can simply emit more commands into the speech sequence.
2. Other speech or pressing control naturally stops this utterance from happening, so it won't talk later if you pressed control to shut NVDA up.

[davidacm]

BreakCommand sounds as a good aproach, but... That ensures that it will work with any synth driver? Even if BreakCommand is not supported in some driver?
Some synth drivers don't support this, or are not accurate in time.

I know, native drivers support it, but there are many external synth drivers used by users.

My current add-on can cancel the timer if another event occurs, even if the user press the control key.

[derekriemer]

#13543

[feerrenrut]

As this automatically speaking of character description doesn't apply just to English character, giving a delay allow user to clearly identify the character from the character description

@LittleStar-VIP I understand. I think there are a few ways to achieve this with speech commands / sequences. The approach you use will depend on the scale of the delay required. Most speech separates concepts (E.G. A role description from the accessible name) with a double space. The synthesizer announces this with a slightly longer pause than between words.

As @derekriemer mentions the BreakCommand is potentially a solution. Either way, I think that speech commands are the right way to solve this issue.

Some synth drivers don't support this, or are not accurate in time.

@davidacm this may be a concern, it would be possible to implement this on the NVDA side generically.

[davidacm]

I started to coding when I said I'd do it, based on my add-on EnhancedPhoneticReading but I hadn't had time to update the guide.

I sent the pr now. You can see it in This link

Use breakCommands could be a good idea, but it will need a distinct way from NVDA to handle the breakCommands and I don't think that it will be changed in short term.

[cary-rowen]

@LittleStar-VIP
Have you tested #13550 and if so, any ideas?

[LittleStar-VIP]

@cary-rowen
Thanks for your reminder, I have tested it and forgot to report to you here.
I think the function that the pr build is demonstrating is satisfactory.

[derekriemer]

I think the user guide should explain how long the delay is with the checkbox.

[derekriemer]

and the what's new.

[seanbudd]

@derekriemer The User Guide currently contains a reference to the delay time:

For example, while reviewing a line by characters, when the letter "b" is read NVDA will say "Bravo" after a 1 second delay.

[derekriemer]

Ah, indeed it does. The what's new doesn't but that's fine.

…

On Tue, Jul 19, 2022 at 5:44 PM Sean Budd ***@***.***> wrote: @derekriemer <https://github.com/derekriemer> The User Guide currently contains a reference to the delay time: For example, while reviewing a line by characters, when the letter "b" is read NVDA will say "Bravo" after a 1 second delay. — Reply to this email directly, view it on GitHub <#13509 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI2FPL4J7ABTLK7WN2IYN3VU44WNANCNFSM5RE44TRQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Derek Riemer: Improving the world, one byte at a time. ⠊⠍⠏⠗⠕⠧⠬ ⠮ ⠸⠺⠂ ⠐⠕ ⠃⠽⠞⠑ ⠁⠞ ⠁ ⠐⠞⠲ Software engineer, Drive web

[MertKaraguzel]

Hi, I'm so glad that the delayed character descriptions feature is included in NVDA.
It would be nice if the ability to read delayed character descriptions in a shorter time could be added to this feature.
Thanks,

[MertKaraguzel]

Hello again, also, one more feature can be added.
It is also the feature of turning off the character feature in character descriptions and hearing only the descriptions.
example: Not a-alfa, not b-bravo.
Only alfa, bravo, Charlie like this.
This feature is available in the Enhanced Character Descriptions add-on.
Thanks.

[bdorer]

Thanks, I sign this user perspective.

[CyrilleB79]

@MertKaraguzelcommented and @bdorer, if you want your comments to be taken into account, you'd better open a new issue or comment in an existing matching open issue (if any).
Indeed, closed/resolved issues are less visible by NVAccess and are not triaged again.