Various pressKey events are failing from testing

[gnarf]

Similar to #46 - some keys are now giving an error when the bot sends the pressKey event. The error looks like this in the nvda.log:

127.0.0.1 - - [28/Mar/2024 09:21:36] "POST /settings HTTP/1.1" 200 -
WARNING - stdout (09:21:36.295) - Thread-2 (5304):
executing gesture "insert+space"
ERROR - stderr (09:21:36.298) - Thread-2 (5304):
127.0.0.1 - - [28/Mar/2024 09:21:36] "POST /presskeys HTTP/1.1" 200 -
ERROR - stderr (09:21:37.321) - Thread-2 (5304):
127.0.0.1 - - [28/Mar/2024 09:21:37] "POST /settings HTTP/1.1" 200 -
WARNING - stdout (09:21:37.328) - Thread-2 (5304):
executing gesture "enter"
WARNING - stdout (09:21:37.330) - Thread-2 (5304):
error executing gesture enter:

The error executing gesture comes from the addon, but there doesn't seem to be any other information leading to the cause of the error.

This error is currently reproduceable by running the automation harness on the alert tests with firefox - the error seems to crop up for both the "press space" and "press enter" in "Focus mode"

[jscholes]

@gnarf I'm not clear on the scope of the issue. Specifically, you wrote:

This error is currently reproduceable by running the automation harness on the alert tests with firefox

Are you saying that the same errors don't happen when running tests against Chrome, and/or other test plans in Firefox and Chrome?

[gnarf]

Added in a line to print the exception from the deeper code:

WARNING - stdout (09:09:42.707) - Thread-3 (8436):
executing gesture "space"
WARNING - stdout (09:09:42.708) - Thread-3 (8436):
error executing gesture space:
WARNING - stdout (09:09:42.710) - Thread-3 (8436):
Traceback (most recent call last):
  File "C:\Users\User\Desktop\2023.3.3\userConfig\addons\CommandSocket\globalPlugins\CommandSocket\handler.py", line 91, in _handle_press_keys_command
    inputCore.manager.executeGesture(gesture)
  File "inputCore.pyc", line 574, in executeGesture
inputCore.NoInputGestureAction

This seems to be an error deep in executeGesture https://github.com/nvaccess/nvda/blob/f67d317304c6b139e2ee88dc86c47d58cd2d75de/source/inputCore.py#L574

I'm not sure this method for doing pressKeys is going to work for all of our uses.

[gnarf]

Are you saying that the same errors don't happen when running tests against Chrome, and/or other test plans in Firefox and Chrome?

It may well be happening with chrome, and other test plans, but it is easily reproducable with firefox / alert tests.

Edit: just tested, it also happens with chrome - same tests

[gnarf]

Based on some local testing - I think you could get this to reproduce locally by switching to focus mode with an insert+space then sending a space or enter presskey while in focus mode - I don't believe it needs any other automation bot setup/etc to test it.

[gnarf]

@jscholes I've been trying to figure out what the deeper cause of this issue might be, but haven't been having any luck with understanding what is going on this deep inside the NVDA code base. I'm not that familiar with python in the first place, but also the organization of what is happening in this method seems very strange. It's weird to me that space/enter seem to work fine in Browse Mode but when in Focus Mode, they both seem to be throwing this exception

[gnarf]

See also w3c/aria-at-app#1060 (comment) - This documents which tests are seeing errors in one of the aria-at test suites. Included a youtube capture of my VM running these tests (with the additional stack trace printing)

All of the failing tests conform to either this inputCore.NoInputGestureAction error, or just simply missing the gesture to execute for arrow keys.

[jscholes]

@gnarf I've tried to reproduce this in the way you suggested, i.e.:

Based on some local testing - I think you could get this to reproduce locally by switching to focus mode with an insert+space then sending a space or enter presskey while in focus mode ...

The code I used is at the end of this comment. Unfortunately I can't get NVDA (v2024.1) to throw an error in focus mode or browse mode, in Firefox or Chrome, when emulating a Space and Enter while the browse mode cursor and system focus are on the "Trigger Alert" button of the Alert Example test case.

So, three questions:

1. Is this a blocker, or have you been able to work around it?
2. If you have been able to work around it, what was your solution?
3. Do you have any other suggestions on how I might go about reproducing it?

Tes Code

I made a sample NVDA global plug-in, which can be dropped into the scratchpad directory under globalPlugins. Note that the scratchpad must be enabled under NVDA Settings -> Advanced -> "Enable loading custom code from Developer Scratchpad directory". On the same screen, you can use the "Open developer scratchpad directory" button to get to the right location.

The global plug-in adds two gestures: NVDA+Control+E, and NVDA+Control+Shift+E, to emulate an Enter or Space keypress respectively, via the same mechanism used by the automation add-on. Two seconds after the gesture has been emulated, a message will be spoken indicating that either the gesture worked or failed, and in theory the exception related to a failed gesture should be logged (although I can't make this happen).

import wx

import globalPluginHandler
import inputCore
import keyboardHandler
from logHandler import log
import ui


class GlobalPlugin(globalPluginHandler.GlobalPlugin):
	def script_doEnter(self, gesture):
		try:
			gesture = keyboardHandler.KeyboardInputGesture.fromName('enter')
			inputCore.manager.emulateGesture(gesture)
			wx.CallLater(2000, ui.message, 'it worked')
		except Exception as e:
			log(e)
			wx.CallLater(2000, ui.message, 'it failed')

	def script_doSpace(self, gesture):
		try:
			gesture = keyboardHandler.KeyboardInputGesture.fromName('space')
			inputCore.manager.emulateGesture(gesture)
			wx.CallLater(2000, ui.message, 'it worked')
		except Exception as e:
			log(e)
			wx.CallLater(2000, ui.message, 'it failed')

	__gestures = {
		'kb:nvda+control+e': 'doEnter',
		'kb:nvda+control+shift+e': 'doSpace',
	}

[jscholes]

@gnarf Looking into NVDA itself, part of the code for inputCore.manager.emulateGesture looks like this:

		try:
			return self.executeGesture(gesture)
		except NoInputGestureAction:
			pass

In other words, the NoInputGestureAction exception is caught and then silently ignored, allowing NVDA to move onto a fallback method for emulating the gesture. However, your traceback in #49 (comment) indicates that NoInputGestureAction is exactly the exception being raised, which according to NVDA's code is impossible.

[gnarf]

So, three questions:

1. Is this a blocker, or have you been able to work around it?
2. If you have been able to work around it, what was your solution?
3. Do you have any other suggestions on how I might go about reproducing it?

It is still causing "empty output" errors when running the tests... I made a clip of my local VM running the test suite for w3c/aria-at-app#1060 (comment) - these should be pretty obvious when they are happening from the logs.

This version of the tests was for tab/shift+tab and some arrow keys that aren't working, but it was the same "root" error - perhaps space and enter got fixed by something else I did but tab/shift+tab are still failing?

I'm not sure

[jscholes]

@gnarf I can't reproduce any error emulating Tab or Shift+Tab, in browse mode or focus mode, in Chrome or Firefox.

[gnarf]

I reran the same suite that had these errors before, and it doesn't seem to happen anymore.... Very confusing. This error was reported on 2023.3, so I'm guessing that 2024.1 fixed that problem with tab/shift+tab and arrows for us without me knowing.

Thanks for looking into it and providing some extra debugging tips for NVDA internals, it will definitely come in handy if I need to dive in on something like this one again!