NVDA does not announce F-keys as Keystrokes of JMenuItems (Java Access Bridge)

When selecting a JMenuItem which has an F-Key in its keystroke (accelerator key), NVDA does not announce the F-key although the information is transmitted via the Java Access Bridge. Other screen readers announce the F-key in a JMenuItem's keystroke. This issue affects also JCheckBoxMenuItem and JRadioButtonMenuItem as sub classes of JMenuItem.

If a keystroke is assigned to a JMenuItem, the user can fire the assigned action directly by hitting the keystroke. To use this quick access, the user needs to know the keystroke.

I wrote a simple application Test_JMenuItem_KeyStrokes.jar with a menu containing five JCheckBoxMenuItems with different keystrokes and attached it to this bug report.

Steps to reproduce:

1. Extract the Test_JMenuItem_KeyStrokes.jar file from the linked .zip file:
   Test_JMenuItem_KeyStrokes.zip
2. Execute the Test_JMenuItem_KeyStrokes.jar file which opens a simple frame with a checkbox and a single menu in the menu bar. (If you prefer a .java file, you can find it within the .jar file.)
   -> A Java window with title "A Frame with Menu" opens.
   -> The checkbox with label "A checkbox" has the keyboard focus.
3. Press F10 to open the single menu.
   -> The menu "Menu with Keystrokes" opens and NVDA announces it and its keystroke Alt+M.
   Note: You can also open this menu with its keystroke Alt+M.
   -> The first menu item is selected and NVDA anounces it.
4. Navigate with arrow down/up keys within the menu.

Actual behavior:

NVDA announces the menu items with its keystrokes as follows (leaving out the announced state information in detail):

• "First Menu Item", its state, "Ctrl a"
• "Second Menu Item", its state, "Ctrl Shift a"
• "Third Menu Item". its state
• "Fourth Menu Item", its state, "Ctrl"
• "Fifth Menu Item", its state, "Ctrl Shift"

Note, that the F-key (F4) is missing in the third, fourth and fifth menu item.

Expected behavior:

For the third, fourth and fifth menu item, NVDA should announce the following:

• "Third Menu Item", its state, "F4"
• "Fourth Menu Item", its state, "Ctrl F4"
• "Fifth Menu Item", its state, "Ctrl Shift F4"

System configuration

NVDA installed/portable/running from source:

installed

NVDA version:

Version: 2021.3.5

Windows version:

Version: 10.0.19042 Build 19042

Name and version of other software in use when reproducing the issue:

java version "18.0.1" 2022-04-19
Java(TM) SE Runtime Environment (build 18.0.1+10-24)
Java HotSpot(TM) 64-Bit Server VM (build 18.0.1+10-24, mixed mode, sharing)

Other information about your system:

Java Access Bridge is enabled.

Other questions

Does the issue still occur after restarting your computer?

Yes.

Have you tried any other versions of NVDA? If so, please report their behaviors.

Yes. Same behavior.

If NVDA add-ons are disabled, is your problem still occurring?

Not applicable.

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Not applicable.

[mwhapples]

I have made a pull request to fix this, see #13676 does that fix the issue for you? There are a few small things I would really like to check:

• Certain keys like space do not seem to be handled by the special keys in java access bridge, so i am not sure how it really is handled.
• There are some modifiers not handled by NVDA. Some may not be relevant to Windows, but others like altgraph certainly are, should NVDA be extended to support these additional modifiers?

Thanks, I would like to test the bug fix. How can I get a test version?

Regarding your questions:

• Space is not a special key. I guess you are asking because of the controlCode(...) method in Java AccessBridge. Space is not a control character, it is a printable character.
• For the modifiers of a KeyStroke, the Java AccessBridge checks in its getModifiers(...) method for the modifiers shift, ctrl, meta and alt. Alt Graph is usually not used as keystroke modifier, but as a switch of a key's printable character. Apart from that, NVDA should be able to announce the Alt Graph key.

[mwhapples]

The build for my original pull request fixing this is at https://ci.appveyor.com/api/buildjobs/fdq2g6d8ya6uw0b8/artifacts/output/nvda_snapshot_pr13676-25373,308a963e.exe but nvaccess made some additional changes to improve the code style but I cannot find a link for a build including their changes.
As for my question about space and some other keys. Seems like Java Access Bridge does not have a good way of communicating those. Although space may be a printable character, looks like may be access bridge was going to include the word "Space" but as it only has a character field it is the first letter of the word (IE. "S"). Some other keys like "Tab" is "T", "Enter" is "E" and "Escape" is "E". So NVDA will just speak these letters for these keys. I think it is a bug in (java Access Bridge and (jaws suffers with the bug for these specific keys as well.
As for the altgraph question, I understand its use as you describe, I think my concern was that it seems like Java will allow it to be used as a modifier, someone probably will and so should NVDA handle it. However as not handling altgraph is a wider NVDA limitation I did not want to go in and change it for NVDA unless others agree. It though probably should be a separate issue on github.

[seanbudd]

You can find a link to the latest build by clicking "Show all Checks" on the bottom of the pull request: https://ci.appveyor.com/project/NVAccess/nvda/builds/43504071/artifacts

Retested with version 2022.2rc1 by running nvda_2022.2rc1.exe:
F-keys (with or without modifier keys) are announced on menu items.
Thanks!