MS Word browse mode: modify the internal keyboard handler so that key down events don't acumulate time since input to the infinite

[Adriani90]

Is your feature request related to a problem? Please describe.

In MS Word documents, when using browse mode, holding down the up and down arrow key, or other navigation related key combinations such as ctrl+up and down arrow keys or selecting with shift+up and down arrow keys, cause NVDA to increase the time since input. The longer you hold down the keys, the higher the TSI will be, theoretically it could go to infinite. The time since input means that after releasing the key, NVDA will need x seconds until the last speech sequence from the keydown event starts to be spoken.
This causes a severe performance issue, especially in large documents.
Moreover, if you end up with a very high time since input, then NVDA will not react to any keyboard actions until all speech sequences from the key down event are completely spoken.

Steps to reproduce

1. Open attached document
   ONKYO.docx
2. With NVDA running, switch to browse mode
3. Hold down up or down arrow key for a longer time (e.g. 1 minute). You can do it with ctrl+arrow keys or shift+arrow keys as well

Actual behavior

Time since input increases continuously with every speech sequence, and NVDA puts speech sequences related to other keyboard actions behind the keydown event. This causes severe performance issues. Following error is written to the log after a while:

ERROR - keyboardHandler.internal_keyDownEvent (23:03:03.134) - winInputHook (26320):
internal_keyDownEvent
Traceback (most recent call last):
  File "keyboardHandler.pyc", line 245, in internal_keyDownEvent
  File "inputCore.pyc", line 539, in executeGesture
  File "queueHandler.pyc", line 51, in queueFunction
  File "core.pyc", line 1031, in requestPump
  File "core.pyc", line 850, in queueRequest
  File "monkeyPatches\wxMonkeyPatches.pyc", line 25, in wx_CallAfter_wrapper
  File "winUser.pyc", line 656, in PostMessage
OSError: [WinError 1816] Not enough quota available to process this command.

Keydown event infinite-speech.txt

Expected behavior

The time since input resets to 0 after every speech sequence, and keys are logged properly like it happens in browse mode in Firefox or Chrome. When the key is released, NVDA speaks imediately the last speech sequence where the virtual cursor is without throwing any error about not enough quota.

Describe the solution you'd like

For browse mode, Change the keyboard and the speech manager handler for MS Word to reset time since input to 0 every time after a speech sequence when holding down a key, similar to what happens in MS Word focus mode or in virtual documents in browsers.

Describe alternatives you've considered

fire speech sequence only at current system carret position after the key is released, and not while the key is held down. This should apply for all kinds of carret navigation and also selection key strokes such as shift+arrow keys.

Additional context

Scrolling the screen should still syncronize between browse and focus mode when holding down a key.

Statistics:

• Holding down up or down arrow key adds up approx. 200-400 ms to the time since input after every speech sequence
• Holding down ctrl+up or ctrl+down arrow key adds up approx. 50-100 ms to the time since input after every speech sequence
• Holding down shift+up or shift+down arrow key adds up approx. 100-250 ms to the time since input after every speech sequence

System info

Asus ROG Strix Intel I12900HK 5 GHZ, Grafic NVIDIA GTX 3070TI 8GB dedicated storage, 32 GB DDR

Enironment

Windows 11 23 H2
NVDA alpha-32650,405f9bfa (2024.3.0.32650)
Microsoft 365 MSO (Version 2406 Build 16.0.17726.20078) 64 Bit

[Adriani90]

cc: @michaelDCurran

[Adriani90]

This happens both with UIA on or off.

[XLTechie]

@Adriani90, do you know when this started? E.g., was it happening last year?

[CyrilleB79]

Just encountered this issue two days ago; it's quite common to hold down arrow or shift+arrow in a document. Doing this cause a severe unavailability of NVDA / application, because holding the key(s) sends many keypresses. Unfortunately, as shown by @Adriani90, each keypress adds an extra 200-300ms (or similar magnitude). Thus the freeze delay can increase very fast.

Even if you do not hold the keypress, pressing about 20 times arrow key or shift+arrow key very quickly, I experiment a freeze / unavailability of 5-10 seconds.

Cc @seanbudd for triaging and priorizing.

[Adriani90]

This is reproducible for years, I just found now time and right words to report it.Von meinem iPhone gesendetAm 04.07.2024 um 10:15 schrieb Luke Davis ***@***.***>: @Adriani90, do you know when this started? E.g., was it happening last year? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[CyrilleB79]

@gerald-hartig you have added papercutslabels. The definition of this label is "minor nuisance, "death by 1000 papercuts"". As I understand it, this label applies to issues that are quite rarely reproducible / reproduced and with low impact; am I correct?

If yes, this does not apply at all to this issue:

• depending on the usage you make, this issue can occur quite frequently, i.e. as soon as you maintain upArrow to go up many lines in an Outlook message for example, which may be quite common.
• The impact isn't so low either since NVDA freeze during many seconds, which is a very bad UX.
• At last, I wonder if the root cause of this issue may explain why NVDA is always slow in complex Outlook messages (newsletters) or Word documents; there are complains about that for years even if UIA has already improved things a bit.

[Adriani90]

@gerald-hartig, @michaelDCurran this is quite severe in my view, it causes NVDA even to crash completely after a while when significant time since input is added to the keydown event.
I think we should reconsider the priority for this issue since MS Office is a very commonly used software.

[gerald-hartig]

@CyrilleB79 @Adriani90 we're rethinking the use of the papercuts label, so please ignore that aspect of the triage. Do you both recommend that this is a P1 rather than a P2? We originally classified it as a P2 as we felt it does not reach the threshold of "A P1 causes the inability to perform a popular task or majority of tasks in NVDA or a popular app."

[Adriani90]

Ah sorry I was not aware of the p2, I think p2 is ok. Thanks for the quick reply.Von meinem iPhone gesendetAm 25.07.2024 um 23:18 schrieb gerald-hartig ***@***.***>: @CyrilleB79 @Adriani90 we're rethinking the use of the papercuts label, so please ignore that aspect of the triage. Do you both recommend that this is a P1 rather than a P2? We originally classified it as a P2 as we felt it does not reach the threshold of "A P1 causes the inability to perform a popular task or majority of tasks in NVDA or a popular app." —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[michaelDCurran]

I can most definitely reproduce this with UIA turned off (object model). Are you absolutely sure this happens with UIA on? On my machine I really need to hold down an arrow key for at least 20 seconds to get something noticeable with UIA on. If so, can you state if you are using braille, or have any particular Document Formatting settings turned on?

I also do want to clarify, that time since input is not some kind of deliberate delay in NVDA, it is mealy reporting how much time has passed between a key being pressed and NVDA starting to speak. "Resetting" some kind of time since input to 0 is not a real thing.
However, we can do things such as in the browse mode caret movement scripts, detect if another script is waiting, and therefore do the least amount of work possible to move the caret, but not bother collecting any extra data in order to report, as the script after will do that anyway.