Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Java SwingSet2 application with NVDA #9728

Closed
DavidApps2 opened this issue Jun 12, 2019 · 5 comments · Fixed by #13744
Closed

Java SwingSet2 application with NVDA #9728

DavidApps2 opened this issue Jun 12, 2019 · 5 comments · Fixed by #13744
Milestone
2022.2

Comments

@DavidApps2
Copy link

I have encountered some issues with the SwingSet2 application, which is a Java Swing application that Oracle suggest is suitable for testing whether the Java Access Bridge is working correctly.

I have listed several issues. Some items might be due to my ignorance. Some items might be due to my high expectations. Some items might be due to inevitable behavior when using the Java Access Bridge or Java Swing applications. Some items might be due to the way the Java Swing application is written. Some items might be NVDA bugs or feature requests. I asked for help with this on the NVDA mailing list, but I did not receive a reply. I am sorry if my report is unconventional.

Steps to reproduce:

These instructions apply to 64-bit Microsoft Windows.

  1. Install the Java Runtime Environment available from the page at the following address:
    https://www.java.com/en/download/manual.jsp
  2. Enable the Java Access Bridge, using the instructions on the page at the following address:
    https://docs.oracle.com/javase/8/docs/technotes/guides/access/enable_and_test.html
    The 32-bit Java Access Bridge DLL (WindowsAccessBridge-32.dll) should be in the sysWOW64 folder (usually C:\Windows\sysWOW64). The 64-bit Java Access Bridge DLL (WindowsAccessBridge-64.dll) should be in the system32 folder (usually C:\Windows\system32).
  3. Download jdk-8u211-windows-x64-demos.zip from the page at the following address:
    https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
  4. Extract the files from the .zip archive.
  5. Start demo\jfc\SwingSet2\SwingSet2.jar with a command of the form java -jar SwingSet2.jar.
  6. Press the Tab key to navigate to the "JButton, JRadioButton, JToggleButton, JCheckbox demos" toggle button. NVDA says "toggle button 0 not pressed JButton, JRadioButton, JToggleButton, JCheckbox demos 2 of 16". Why does NVDA say "0"? Doing so does not seem useful here.
  7. Press the Space key to activate the toggle button. NVDA says "1 checked". Why does NVDA say "1"? Doing so does not seem useful here. Saying "pressed" might be more helpful than "1 checked". (This is likely to be related to item 6.)
  8. NVDA now reads the toggle button as "toggle button 1 not pressed checked JButton, JRadioButton, JToggleButton, JCheckbox demos 2 of 16". Why does NVDA say "1 not pressed" when "pressed" would probably be more helpful? (This might be related to items 6 and 7.)
  9. Press the Tab key until you reach the "Three!" button. Press NVDA+Up Arrow quickly to get NVDA to spell the label of the button. NVDA spells three spaces before the button label. Is this what you would expect?
  10. In the "Image Buttons" group, should NVDA read the image buttons as "button"? (Perhaps the images have no alternative text labels.)
  11. In the "Display Options" group of check boxes and the "Pad Amount" group of radio buttons, should NVDA read the "Display Options" or "Pad Amount" group label? I expected to hear the group label as I use Tab or Shift+Tab to enter the group, but I do not hear it. Perhaps the group label is not correctly associated with the check boxes or the radio buttons.
  12. Press the "JColorChooser demo" toggle button. NVDA stops responding. After restarting NVDA, NVDA has no access to the Java Swing application. Is there anything that the user or the developer of the application can do to re-establish access? The only method that I have found is exiting the Java Swing application and starting it again. Is there anything that NVDA can do to establish access with a Java application that is already running?
  13. Press the "JOptionPane Demo" toggle button.
  14. Press the "Show Input Dialog" button. A dialog box appears. NVDA does not read the label of the edit field. When I press NVDA+B, NVDA does not properly read the dialog box. After pressing Alt+Tab twice to switch to another application and back again, I can use NVDA+B to learn that I am being asked "What is your favorite movie?". Should NVDA read the label or is it not correctly associated with the edit field?
  15. Enter some text in the edit field and press the "OK" button. A message box appears. When it does so, NVDA reads the message, but NVDA+B does not get NVDA to read the message to me again. Should it? If I use Alt+Tab twice to switch to another application and back again, NVDA+B reads the message box to me, as expected.
  16. Press the "JTabbedPane demo" toggle button.
  17. The tab control has four tabs with the labels "Hania", "Ewan", "Laine", and "Bouncing Babies!". Should NVDA report the number of the tab and the number of tabs (for example, "tab 1 of 4")? This would be useful, because the tab panel control wraps.
  18. Select the "Bouncing Babies!" tab and press NVDA+Up Arrow twice quickly to get NVDA to spell the label of the tab. NVDA spells four spaces at the start of the label. Is this what you would expect? (This might be related to item 9.)
  19. Press the "JTable demo" button. In the "Favorite Food" column of the table, NVDA does not read the contents of the column. (Perhaps the images have no alternative text labels.) (This might be related to item 10.)

System configuration

NVDA installed/portable/running from source:

Installed

NVDA version:

2019.1.1

Windows version:

10.0.17763 Build 17763

Other questions

Does the issue still occur after restarting your PC?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

Same behavior with NVDA 2018.3 (with earlier versions of the Java runtime environment).
nvda-old2.log
nvda-old.log
nvda.log
@mwhapples
Copy link
Collaborator

I have tried the above with NVDA 2022.1, Java 17 and the demos from JDK17. Here are my findings:

  • Items 6, 7 and 8 I observe the problem. I have made a fix and will be submitting it when I fix other parts of this issue.
  • 9 I observe no spaces. However it may have been different in JDK8 demos, see my comments relating to item 18.
  • 10 I think this may be a limitation of what is presented through Java Access Bridge. I have checked what is available with AccessBridgeExplorer.
  • 11 the label is not part of a gouping pane and no accessible relationship is set in the demo source code.
  • 12 I do not observe with JDK17 demos.
  • 14 no accessible relationship between label and edit box. I will try and fix the issue with reading the window.
  • 15 will try and fix the read window issue.
  • 17 native windows tabs, such as those in file properties dialogs in windows explorer, do not have this information. It may be possible to get this through Java Access Bridge and I could try and do that if there is enough interest.
  • 18 Java controls can be formatted by using HTML tags. At the moment NVDA replaces HTML tags with spaces. These spaces are due to HTML being used for this control. We could just ignore HTML tags rather than replace with a space.
  • 19 Appears to work for me when using JDK 17 demos.
    As I said I have a fix for the toggle buttons, I will try and fix the read window thing as that is clearly incorrect. Let me know if you want me to try and give the additional information for tab controls and if to make HTML tags ignored rather than replaced with spaces.

@DavidApps2
Copy link
Author

Thank you very much for looking at these issues and for fixing what you can. It sounds as if you have looked at these issues very thoroughly. Thank you.

It would be great if you can fix the read window issue in items 14 and 15.

With item 17, these tabs wrap. The tabs that you mention in the properties dialog box wrap when I use Ctrl+Tab and Ctrl+Shift+Tab but do not wrap when I use Left Arrow and Right Arrow. I think that the extra contextual information is more useful when the tabs wrap. When the tabs wrap, I find it much less obvious that I have worked my way through all the tabs. This is probably much less of an issue for people with good short-term memory who can more easily recognize when they have returned to the tab that they started on. The need for extra contextual information is less when the tabs do not wrap or when the number of tabs is smaller. A user who can see the screen can tell visually when the current tab has moved from one end of the tab list to the other. Perhaps you can ask some of the other users you know to get an idea of whether many people are interested in having this contextual information.

With item 18, perhaps NVDA replaces an HTML tag with a space to avoid joining separate words. Would it be a lot of work to ignore the tags at the beginning and end of the control and before or after a space so as to avoid starting or ending with a space or inserting multiple spaces?

Thank you.

@mwhapples
Copy link
Collaborator

An update on this.

  • I now have a fix for the read window issues from step 14 and 15. I will submit it once I have dealt with the remaining issues in this issue report.
  • On the HTML tags, thanks for reminding me why inserting a space may have been done, to prevent joining words together. I should be able to make the HTML processing smarter so that it only inserts a space between words where no space exists. However that potentially could lead to splitting of part emphasised words, which the current processing would do anyway. However I think that is rare enough and it would still be no worse than what currently is done that I feel doing as you suggest is a good fix for now.
  • On the tabs thing, having looked at other applications (eg. thunderbird, google chrome, etc) there are some examples where the information you are asking for is provided by NVDA. I think Java Access Bridge does give enough information so I will try and fix it or provide an update here if I hit problems.

@mwhapples mwhapples mentioned this issue May 27, 2022
Merged
5 tasks
@mwhapples
Copy link
Collaborator

I have fixed the things I had identified as needing fixing in the JDK17 demos with NVDA 2022.1. There is now a pull request #13744 for this should you want to test my fixes.

@DavidApps2
Copy link
Author

Thank you very much. That sounds great.

seanbudd pushed a commit that referenced this issue Jun 2, 2022
@mwhapples
Java Access Bridge announcement fixes: toggle buttons, window titles,…
12e17fc 
… position information (#13744)

Fixes #9184 and fixes #9728 

Summary of the issue:
A number of features in Java Access Bridge were not working optimally.
1. Various commands like read window and read window title were not working when Java applications have multiple windows. (#9184)
2. Toggle buttons were not read correctly. (#9728)
3. Unnecessary spaces when HTML tags removed from Java controls. (#9728)
4. Provide position information for Java tab controls. (#9728)

Description of how this pull request fixes the issue:
1. According to Java documentation* when getAccessibleIndexInParent returns -1 there is no parent. So in NVDA if indexInParent is None we should treat it as if parent is not set.
2. Java toggle buttons use the checked state for pressed, correctly map this. The value should be ignored for a toggle button.
3. Updated the regular expression processing for HTML tags to check if the tags are at the start or end or have surrounding whitespace, in which case the tag is stripped. If there is only non-whitespace characters around the tag then replace the tag with a space to prevent words being joined.
4. For Java controls with Role.TAB use indexInParent and parent.childCount to find the position information.

* https://docs.oracle.com/en/java/javase/17/docs/api/java.desktop/javax/accessibility/AccessibleContext.html#getAccessibleIndexInParent()
@nvaccessAuto nvaccessAuto modified the milestone: 2022.2 Jun 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
2022.2
3 participants
@mwhapples @nvaccessAuto @DavidApps2