Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

8267385: Create NSAccessibilityElement implementation for JavaComponentAccessibility #4412

Closed
wants to merge 15 commits into from
Closed

8267385: Create NSAccessibilityElement implementation for JavaComponentAccessibility #4412

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
c05c814
8267385: Create NSAccessibilityElement implementation for JavaCompone…
savoptik Jun 8, 2021
66ce693
8262031: Create implementation for NSAccessibilityNavigableStaticText…
savoptik May 30, 2021
f7979c6
8264287: Create implementation for NSAccessibilityComboBox protocol peer
savoptik May 30, 2021
e8b284f
8264303: Create implementation for NSAccessibilityTabGroup protocol peer
savoptik May 30, 2021
dd219ab
8264292: Create implementation for NSAccessibilityList protocol peer
savoptik May 30, 2021
d74af05
8267387: Create implementation for NSAccessibilityOutline protocol
savoptik May 30, 2021
280ee5f
8267388: Create implementation for NSAccessibilityTable protocol
savoptik May 30, 2021
e9b966b
Even on Linux? I meant if the test will be run on the platform where …
savoptik Jul 2, 2021
ba09569
1. Fixed memory management problems, due to which a11y cells in table…
savoptik Jul 13, 2021
fbe3443
Why not mark them as @Native in the CAccessibility and use constants …
savoptik Jul 14, 2021
76cba14
ok, so this is in the new code, how hard it will be to reimplement th…
savoptik Jul 15, 2021
b4ef57a
1. Removing the warning about an unaccessible element in the console.
savoptik Jul 20, 2021
a07c86d
Wildcard imports are avoided in new tests
savoptik Jul 20, 2021
6505334
1. The comment states that it is not necessary to call ExceptionCheck…
savoptik Jul 21, 2021
a13d6a6
1. Fixed problems with vertical navigation in table cells;
savoptik Aug 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
227 changes: 166 additions & 61 deletions src/java.desktop/macosx/classes/sun/lwawt/macosx/CAccessibility.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,13 @@
import java.awt.event.KeyEvent;
import java.beans.PropertyChangeEvent;
import java.beans.PropertyChangeListener;
import java.lang.annotation.Native;
import java.lang.reflect.InvocationTargetException;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.Callable;
import java.util.Arrays;

import javax.accessibility.Accessible;
import javax.accessibility.AccessibleAction;
Expand All @@ -57,6 +59,8 @@
import javax.swing.JLabel;
import javax.swing.JMenuItem;
import javax.swing.JTextArea;
import javax.swing.JList;
import javax.swing.JTree;
import javax.swing.KeyStroke;

import sun.awt.AWTAccessor;
Expand Down Expand Up @@ -116,12 +120,7 @@ public void propertyChange(final PropertyChangeEvent evt) {
private native void focusChanged();

static <T> T invokeAndWait(final Callable<T> callable, final Component c) {
if (c != null) {
try {
return LWCToolkit.invokeAndWait(callable, c);
} catch (final Exception e) { e.printStackTrace(); }
}
return null;
return invokeAndWait(callable, c, null);
}

static <T> T invokeAndWait(final Callable<T> callable, final Component c, final T defValue) {
Expand Down Expand Up @@ -551,6 +550,10 @@ public void run() {
if (pac == null) return;
AccessibleSelection as = pac.getAccessibleSelection();
if (as == null) return;
if (parent instanceof JList) {
((JList) parent).setSelectedIndex(i);
return;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the a11y interface miss "setSelectedIndex" method? The code above will not work for the custom component which uses the "AccessibleJListChild" a11y object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, the current a11y interface does not allow this action.
As for the custom AccessibleJListChild, I prepared a sample project in which I implemented a custom child. The experiment showed that there are no negative consequences. The custom child works as intended.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the example, you use a JList class, and the checks in the code above works, but if some other "AccessibleJList" class will be used then it will not work, right? This is the reason why implementations of a11y classes usually use specific interfaces like "AccessibleJList".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this will not work with a custom list not inherited from JList. Unfortunately AccessibleJList is not perfect and the functionality you need is not there.
If you have a better solution, please share it. - Could you please advice on what we can do here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I meant in the first message, "a11y interface miss setSelectedIndex method". I suggest filing a separate issue to accumulate such use cases and then fix them by adding additional methods to a11y interfaces. Probably something similar will be found in the Table/etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. JDK-8271846

as.addAccessibleSelection(i);
}
}, c);
Expand Down Expand Up @@ -651,77 +654,148 @@ public boolean[] call() throws Exception {

// Duplicated from JavaComponentAccessibility
// Note that values >=0 are indexes into the child array
static final int JAVA_AX_ALL_CHILDREN = -1;
static final int JAVA_AX_SELECTED_CHILDREN = -2;
static final int JAVA_AX_VISIBLE_CHILDREN = -3;
@Native static final int JAVA_AX_ALL_CHILDREN = -1;
@Native static final int JAVA_AX_SELECTED_CHILDREN = -2;
@Native static final int JAVA_AX_VISIBLE_CHILDREN = -3;

// Each child takes up two entries in the array: one for itself and one for its role
public static Object[] getChildrenAndRoles(final Accessible a, final Component c, final int whichChildren, final boolean allowIgnored) {
if (a == null) return null;
return invokeAndWait(new Callable<Object[]>() {
public Object[] call() throws Exception {
ArrayList<Object> childrenAndRoles = new ArrayList<Object>();
_addChildren(a, whichChildren, allowIgnored, childrenAndRoles);

/* In the case of fetching a selection, need to check to see if
* the active descendant is at the beginning of the list. If it
* is not it needs to be moved to the beginning of the list so
* VoiceOver will annouce it correctly. The list returned
* from Java is always in order from top to bottom, but when shift
* selecting downward (extending the list) or multi-selecting using
* the VO keys control+option+command+return the active descendant
* is not at the top of the list in the shift select down case and
* may not be in the multi select case.
*/
if (whichChildren == JAVA_AX_SELECTED_CHILDREN) {
if (!childrenAndRoles.isEmpty()) {
AccessibleContext activeDescendantAC =
CAccessible.getActiveDescendant(a);
if (activeDescendantAC != null) {
String activeDescendantName =
activeDescendantAC.getAccessibleName();
AccessibleRole activeDescendantRole =
activeDescendantAC.getAccessibleRole();
// Move active descendant to front of list.
// List contains pairs of each selected item's
// Accessible and AccessibleRole.
ArrayList<Object> newArray = new ArrayList<Object>();
int count = childrenAndRoles.size();
Accessible currentAccessible = null;
AccessibleContext currentAC = null;
String currentName = null;
AccessibleRole currentRole = null;
for (int i = 0; i < count; i+=2) {
// Is this the active descendant?
currentAccessible = (Accessible)childrenAndRoles.get(i);
currentAC = currentAccessible.getAccessibleContext();
currentName = currentAC.getAccessibleName();
currentRole = (AccessibleRole)childrenAndRoles.get(i+1);
if (currentName != null && currentName.equals(activeDescendantName) &&
currentRole.equals(activeDescendantRole) ) {
newArray.add(0, currentAccessible);
newArray.add(1, currentRole);
} else {
newArray.add(currentAccessible);
newArray.add(currentRole);
}
}
childrenAndRoles = newArray;
return getChildrenAndRolesImpl(a, c, whichChildren, allowIgnored);
}
}, c);
}

private static Object[] getChildrenAndRolesImpl(final Accessible a, final Component c, final int whichChildren, final boolean allowIgnored) {
if (a == null) return null;

ArrayList<Object> childrenAndRoles = new ArrayList<Object>();
_addChildren(a, whichChildren, allowIgnored, childrenAndRoles);

/* In case of fetching a selection, we need to check if
* the active descendant is at the beginning of the list, or
* otherwise move it, so that VoiceOver announces it correctly.
* The java list is always in order from top to bottom, but when
* (1) shift-selecting downward (extending the list) or (2) multi-selecting with
* the VO keys (CTRL+ALT+CMD+RETURN) the active descendant
* is not at the top of the list in the 1st case and may not be in the 2nd.
*/
if (whichChildren == JAVA_AX_SELECTED_CHILDREN) {
if (!childrenAndRoles.isEmpty()) {
AccessibleContext activeDescendantAC =
CAccessible.getActiveDescendant(a);
if (activeDescendantAC != null) {
String activeDescendantName =
activeDescendantAC.getAccessibleName();
AccessibleRole activeDescendantRole =
activeDescendantAC.getAccessibleRole();
// Move active descendant to front of list.
// List contains pairs of each selected item's
// Accessible and AccessibleRole.
ArrayList<Object> newArray = new ArrayList<Object>();
int count = childrenAndRoles.size();
Accessible currentAccessible = null;
AccessibleContext currentAC = null;
String currentName = null;
AccessibleRole currentRole = null;
for (int i = 0; i < count; i += 2) {
// Is this the active descendant?
currentAccessible = (Accessible) childrenAndRoles.get(i);
currentAC = currentAccessible.getAccessibleContext();
currentName = currentAC.getAccessibleName();
currentRole = (AccessibleRole) childrenAndRoles.get(i + 1);
if (currentName != null && currentName.equals(activeDescendantName) &&
currentRole.equals(activeDescendantRole)) {
newArray.add(0, currentAccessible);
newArray.add(1, currentRole);
} else {
newArray.add(currentAccessible);
newArray.add(currentRole);
}
}
childrenAndRoles = newArray;
}
}
}

if ((whichChildren < 0) || (whichChildren * 2 >= childrenAndRoles.size())) {
return childrenAndRoles.toArray();
}

return new Object[]{childrenAndRoles.get(whichChildren * 2), childrenAndRoles.get((whichChildren * 2) + 1)};
}

// This method is called from the native
// Each child takes up three entries in the array: one for itself, one for its role, and one for the recursion level
private static Object[] getChildrenAndRolesRecursive(final Accessible a, final Component c, final int whichChildren, final boolean allowIgnored, final int level) {
if (a == null) return null;
return invokeAndWait(new Callable<Object[]>() {
public Object[] call() throws Exception {
ArrayList<Object> currentLevelChildren = new ArrayList<Object>();
ArrayList<Object> allChildren = new ArrayList<Object>();
ArrayList<Accessible> parentStack = new ArrayList<Accessible>();
parentStack.add(a);
ArrayList<Integer> indexses = new ArrayList<Integer>();
Integer index = 0;
int currentLevel = level;
while (!parentStack.isEmpty()) {
Accessible p = parentStack.get(parentStack.size() - 1);

currentLevelChildren.addAll(Arrays.asList(getChildrenAndRolesImpl(p, c, JAVA_AX_ALL_CHILDREN, allowIgnored)));
if ((currentLevelChildren.size() == 0) || (index >= currentLevelChildren.size())) {
if (!parentStack.isEmpty()) parentStack.remove(parentStack.size() - 1);
if (!indexses.isEmpty()) index = indexses.remove(indexses.size() - 1);
currentLevel -= 1;
currentLevelChildren.clear();
continue;
}

Accessible ca = null;
Object obj = currentLevelChildren.get(index);
if (!(obj instanceof Accessible)) {
index += 2;
currentLevelChildren.clear();
continue;
}
ca = (Accessible) obj;
Object role = currentLevelChildren.get(index + 1);
currentLevelChildren.clear();

AccessibleContext cac = ca.getAccessibleContext();
if (cac == null) {
index += 2;
continue;
}

if ((cac.getAccessibleStateSet().contains(AccessibleState.SELECTED) && (whichChildren == JAVA_AX_SELECTED_CHILDREN)) ||
(cac.getAccessibleStateSet().contains(AccessibleState.VISIBLE) && (whichChildren == JAVA_AX_VISIBLE_CHILDREN)) ||
(whichChildren == JAVA_AX_ALL_CHILDREN)) {
allChildren.add(ca);
allChildren.add(role);
allChildren.add(String.valueOf(currentLevel));
}

index += 2;

if (cac.getAccessibleStateSet().contains(AccessibleState.EXPANDED)) {
parentStack.add(ca);
indexses.add(index);
index = 0;
currentLevel += 1;
continue;
}

if ((whichChildren < 0) || (whichChildren * 2 >= childrenAndRoles.size())) {
return childrenAndRoles.toArray();
}

return new Object[] { childrenAndRoles.get(whichChildren * 2), childrenAndRoles.get((whichChildren * 2) + 1) };
return allChildren.toArray();
}
}, c);
}

private static final int JAVA_AX_ROWS = 1;
private static final int JAVA_AX_COLS = 2;
@Native private static final int JAVA_AX_ROWS = 1;
@Native private static final int JAVA_AX_COLS = 2;

public static int getTableInfo(final Accessible a, final Component c,
final int info) {
Expand All @@ -740,6 +814,23 @@ public static int getTableInfo(final Accessible a, final Component c,
}, c);
}

private static int[] getTableSelectedInfo(final Accessible a, final Component c,
final int info) {
if (a == null) return null;
return invokeAndWait(() -> {
AccessibleContext ac = a.getAccessibleContext();
AccessibleTable table = ac.getAccessibleTable();
if (table != null) {
if (info == JAVA_AX_COLS) {
return table.getSelectedAccessibleColumns();
} else if (info == JAVA_AX_ROWS) {
return table.getSelectedAccessibleRows();
}
}
return null;
}, c);
}

private static AccessibleRole getAccessibleRoleForLabel(JLabel l, AccessibleRole fallback) {
String text = l.getText();
if (text != null && text.length() > 0) {
Expand Down Expand Up @@ -858,4 +949,18 @@ public Long call() throws Exception {
}
}, (Component)ax);
}

private static boolean isTreeRootVisible(Accessible a, Component c) {
if (a == null) return false;

return invokeAndWait(new Callable<Boolean>() {
public Boolean call() throws Exception {
Accessible sa = CAccessible.getSwingAccessible(a);
if (sa instanceof JTree) {
return ((JTree) sa).isRootVisible();
}
return false;
}
}, c);
}
}
13 changes: 9 additions & 4 deletions src/java.desktop/macosx/classes/sun/lwawt/macosx/CAccessible.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,7 @@

import javax.accessibility.Accessible;
import javax.accessibility.AccessibleContext;
import javax.swing.JProgressBar;
import javax.swing.JTabbedPane;
import javax.swing.JSlider;
import javax.swing.event.ChangeEvent;
import javax.swing.event.ChangeListener;

import static javax.accessibility.AccessibleContext.ACCESSIBLE_ACTIVE_DESCENDANT_PROPERTY;
import static javax.accessibility.AccessibleContext.ACCESSIBLE_CARET_PROPERTY;
Expand Down Expand Up @@ -75,6 +71,8 @@ public static CAccessible getCAccessible(final Accessible a) {
private static native void menuOpened(long ptr);
private static native void menuClosed(long ptr);
private static native void menuItemSelected(long ptr);
private static native void treeNodeExpanded(long ptr);
private static native void treeNodeCollapsed(long ptr);

private Accessible accessible;

Expand Down Expand Up @@ -137,6 +135,13 @@ public void propertyChange(PropertyChangeEvent e) {
if (parentAccessible != null) {
parentRole = parentAccessible.getAccessibleContext().getAccessibleRole();
}

if (newValue == AccessibleState.EXPANDED) {
treeNodeExpanded(ptr);
} else if (newValue == AccessibleState.COLLAPSED) {
treeNodeCollapsed(ptr);
}

// At least for now don't handle combo box menu state changes.
// This may change when later fixing issues which currently
// exist for combo boxes, but for now the following is only
Expand Down
Loading