Skip to content

Conversation

@savoptik
Copy link
Contributor

@savoptik savoptik commented May 6, 2022

A11Y implementation on macOS has to directly call the 'JList.setSelectedIndex' method in order to request selection on an item (see 'CAccessibility.requestSelection'). The reason is that a11y API lacks appropriate method.There's only 'javax.accessibility.AccessibleSelection#addAccessibleSelection' which is mapped to 'javax.swing.JList#addSelectionInterval', it can not be used to set selected index.

@forantar @azuev-java @mrserb please review.

Please note that the new API allows you to implement a multiple selection in lists from the Java side, but I did not succeed in implementing it, because I could not determine the inclusion of the so-called "VoiceOver multiple selection mode".


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change requires a CSR request matching fixVersion 20 to be approved (needs to be created)
  • Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk pull/8578/head:pull/8578
$ git checkout pull/8578

Update a local copy of the PR:
$ git checkout pull/8578
$ git pull https://git.openjdk.org/jdk pull/8578/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 8578

View PR using the GUI difftool:
$ git pr show -t 8578

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/8578.diff

@bridgekeeper
Copy link

bridgekeeper bot commented May 6, 2022

👋 Welcome back asemenov! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented May 6, 2022

@savoptik The following label will be automatically applied to this pull request:

  • client

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added client client-libs-dev@openjdk.org rfr Pull request is ready for review labels May 6, 2022
@mlbridge
Copy link

mlbridge bot commented May 6, 2022

Webrevs

/**
* This interface provides list specific data.
*
* @author Artem Semenov
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't do @author tags in openjdk

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -0,0 +1,223 @@
/*
* Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
* Copyright (c) 2021, JetBrains s.r.o.. All rights reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 2022 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@prrace
Copy link
Contributor

prrace commented May 6, 2022

/csr

@prrace
Copy link
Contributor

prrace commented May 6, 2022

/reviewers 2

@openjdk openjdk bot added the csr Pull request needs approved CSR before integration label May 6, 2022
@openjdk
Copy link

openjdk bot commented May 6, 2022

@prrace has indicated that a compatibility and specification (CSR) request is needed for this pull request.
@savoptik please create a CSR request for issue JDK-8271846. This pull request cannot be integrated until the CSR request is approved.

@prrace
Copy link
Contributor

prrace commented May 6, 2022

Since you are propsosing new API, you can't backport this. I hope that is understood.

@openjdk
Copy link

openjdk bot commented May 6, 2022

@prrace
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with 1 of role reviewers, 1 of role authors).

@savoptik
Copy link
Contributor Author

/addissue 8286674

@openjdk
Copy link

openjdk bot commented May 12, 2022

@savoptik Unknown command addissue - for a list of valid commands use /help.

@savoptik
Copy link
Contributor Author

/add issue 8286674

@openjdk
Copy link

openjdk bot commented May 12, 2022

@savoptik Unknown command add - for a list of valid commands use /help.

@savoptik
Copy link
Contributor Author

/help

@openjdk
Copy link

openjdk bot commented May 12, 2022

@savoptik Available commands:

  • cc - add or remove an additional classification label
  • clean - Mark the backport pull request as a clean backport
  • contributor - adds or removes additional contributors for a PR
  • covered - used when employer has signed the OCA
  • csr - require a compatibility and specification request (CSR) for this pull request
  • help - shows this text
  • integrate - performs integration of the changes in the PR
  • issue - edit the list of issues that this PR solves
  • jep - require a JDK Enhancement Proposal (JEP) for this pull request
  • label - add or remove an additional classification label
  • open - Set the pull request state to "open"
  • reviewer - manage additional reviewers for a PR
  • reviewers - set the number of additional required reviewers for this PR
  • signed - used after signing the OCA
  • solves - edit the list of issues that this PR solves
  • sponsor - performs integration of a PR that is authored by a non-committer
  • summary - updates the summary in the commit message
  • test - used to run tests

@savoptik
Copy link
Contributor Author

/issue add 8286674

@openjdk
Copy link

openjdk bot commented May 12, 2022

@savoptik
Adding additional issue to issue list: 8286674: New interface for accessible list.

@bridgekeeper
Copy link

bridgekeeper bot commented Jun 10, 2022

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

Just some comment so that the pull request doesn't close.

@bridgekeeper
Copy link

bridgekeeper bot commented Jul 8, 2022

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

savoptik commented Jul 8, 2022

Comment to prevent the pull request from closing.

@savoptik
Copy link
Contributor Author

@azuev-java @prrace please review this CSR ticket JDK-8286674

@prrace
Copy link
Contributor

prrace commented Jul 15, 2022

@azuev-java @prrace please review this CSR ticket JDK-8286674

Before we get to that

  1. You did not acknowledge this is not a backportable fix
  2. I'd like you to explain why calling setSelectedIndex isn't good enough AND why Windows A11Y does not need this API
  3. It is incorrect to add the csr as an issue - skara tooling has made it an integration blocker.

@savoptik
Copy link
Contributor Author

savoptik commented Aug 1, 2022

/issue help

@openjdk
Copy link

openjdk bot commented Aug 1, 2022

@savoptik Command syntax:

  • /issue [add|remove] <id>[,<id>,...]
  • /issue [add] <id>: <description>
  • `/issue create [PX] /[subcomponent]

Some examples:

  • /issue add JDK-1234567,4567890
  • /issue remove JDK-4567890
  • /issue 1234567: Use this exact title
  • `/issue create hotspot/jfr
  • `/issue create P4 core-libs/java.nio

If issues are specified only by their ID, the title will be automatically retrieved from JBS. The project prefix (JDK- in the above examples) is optional. Separate multiple issue IDs using either spaces or commas.

@savoptik
Copy link
Contributor Author

savoptik commented Aug 1, 2022

/issue remove 8286674

@savoptik
Copy link
Contributor Author

savoptik commented Aug 2, 2022

@prrace Thanks a lot.
I updated the CSR ticket, and added information there, including your question.

@bridgekeeper
Copy link

bridgekeeper bot commented Aug 30, 2022

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

@azuev-java @mrserb @prrace please review CSR and this PR.

@bridgekeeper
Copy link

bridgekeeper bot commented Sep 28, 2022

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

No, you don't need to close this pull request.
@azuev-java @mrserb @prrace please review CSR and this PR.

@bridgekeeper
Copy link

bridgekeeper bot commented Oct 26, 2022

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

No, you don't need to close this pull request.
@azuev-java @mrserb @prrace please review CSR and this PR.

@prrace
Copy link
Contributor

prrace commented Nov 7, 2022

I am still waiting for the following questions (from July) to be answered here (not somewhere else)

You did not acknowledge this is not a backportable fix

By which I mean, if this needs to be solved in say JDK 11 too, what will you do ??

I'd like you to explain why calling setSelectedIndex isn't good enough AND why Windows A11Y does not need this API

Also leaving aside that you haven't yet shown the API is needed I note that there are javax.accessibility is NOT
the place for Component specific A11Y classes.
So adding javax.accessibility.AccessibleList looks like a very anomalous and perhaps incorrect design.

The similar examples are nested classes of the component, eg
JTree.AccessibleJTree
JTable.AccessibleJTable

In fact there's even already a JList.AccessibleJList !

And it seems to have things that look similar to things you say you need
https://docs.oracle.com/en/java/javase/17/docs/api/java.desktop/javax/swing/JList.AccessibleJList.html#isAccessibleChildSelected(int)

and your API

  • boolean isSelectedIndex(int index);

===

and
public int getAccessibleSelectionCount()
Returns the number of items currently selected. If no items are selected, the return value will be 0.

===
and you have

  • /**
  • * Returns true if no indices are selected.
    
  • *
    
  • * @return {@code true} if no indices are selected.
    
  • */
    
  • boolean isSelectionEmpty();

etc etc ..

@savoptik
Copy link
Contributor Author

savoptik commented Nov 8, 2022

@prrace
I acknowledge that this change is not portable.

The absence of the dedicated AccessibleList interface does not allow setting selections on lists on MacOS using the accessible shortcuts and external accessibility devices. The current solution severely limits the ability of accessibility subsystem to interact with the list component thus it is not acceptable. Existing implementation uses AccessibleSelection implemented in the JList but it lacks ability to work with multiple selection intervals and side effects of setting singular selection with sequential calling of clearSelection() and addSelection() lead to the the line selection action being repeated by the VoiceOver indefinitely.

We need a new accessibility interface that will implement the selection model for accessibility lists. Also it should provide additional information about the list, regardless of whether the accessibility list is inherited from JList or impolemented from scratch. New interface should be able to provide system with the additional information such as selection mode and should be flexible enough to allow implementation of both single and multiselection modes. The AccessibleSelection interface does not allow working with index intervals, which will prevent the implementation of multiple selection in the future.
This interface is not yet relevant on windows because the most common readers do not have a screen navigation modes similar to VoiceOver quick navigation and the current functionality is sufficient.

import javax.accessibility.AccessibleStateSet;
import javax.accessibility.AccessibleText;
import javax.accessibility.AccessibleValue;
import java.util.*;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are trying not to use wildcard imports - probably automatic optimization of the IDE did it? I am always turning it off or set margin so high it is not triggered on anything with less then 200 individual package imports.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

* @test
* @bug 8271846
* @summary Test implementation of AccessibleList interface
* @author Artem.Semenov@jetbrains.com
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not using author tags in tests either - but that's just a nitpick.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@prrace
Copy link
Contributor

prrace commented Nov 9, 2022

@prrace I acknowledge that this change is not portable.

The absence of the dedicated AccessibleList interface does not allow setting selections on lists on MacOS using the accessible shortcuts and external accessibility devices. The current solution severely limits the ability of accessibility subsystem to interact with the list component thus it is not acceptable. Existing implementation uses AccessibleSelection implemented in the JList but it lacks ability to work with multiple selection intervals and side effects of setting singular selection with sequential calling of clearSelection() and addSelection() lead to the the line selection action being repeated by the VoiceOver indefinitely.

We need a new accessibility interface that will implement the selection model for accessibility lists. Also it should provide additional information about the list, regardless of whether the accessibility list is inherited from JList or impolemented from scratch. New interface should be able to provide system with the additional information such as selection mode and should be flexible enough to allow implementation of both single and multiselection modes. The AccessibleSelection interface does not allow working with index intervals, which will prevent the implementation of multiple selection in the future. This interface is not yet relevant on windows because the most common readers do not have a screen navigation modes similar to VoiceOver quick navigation and the current functionality is sufficient.

You didn't speak to the duplicated functionality.
We should retrofit / add to the existing nested class, if new API is truly needed.
"Nothing here is "inherited from JList". AccessibleJList is a nested class, not an inheritance relationship.
API mistakes are hard to un-do so it is not possible for me to sign off on this without somehow finding
enough time to study this API and come to my own conclusions since what I see now looks so anomalous.
Perhaps if you can find someone who knows the A11Y APIs it would go faster, but that isn't me.

@savoptik
Copy link
Contributor Author

savoptik commented Nov 9, 2022

@prrace in file src/java.desktop/macosx/classes/sun/lwawt/macosx/CAccessibility.java:569
There is a direct inheritance dependence on JList. And that is exactly what needs to be avoided. It was pointed out to me here. Without changes, a non-JList-based list will not work.

…zation of the IDE did it? I am always turning it off or set margin so high it is not triggered on anything with less then 200 individual package imports.
@savoptik savoptik force-pushed the asemenov/JDK-8271846 branch from 4a9bc8e to ff82513 Compare November 9, 2022 09:33
@openjdk-notifier
Copy link

@savoptik Please do not rebase or force-push to an active PR as it invalidates existing review comments. All changes will be squashed into a single commit automatically when integrating. See OpenJDK Developers’ Guide for more information.

((AccessibleList) pat).setSelectionInterval(i, i);
return;
}
as.addAccessibleSelection(i);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to clarify one initial question. Why we cannot use clearAccessibleSelection to clean the current selection and then add a new one by the addAccessibleSelection?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tried to do this, it leads to an endless repetition of the selected line until it freezes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably because of the endless notifications loop? implementation of addSelectionInterval looks similar to setSelectionInterval. In some cases the addSelectionInterval just call setSelectionInterval.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like it is. Please formulate the question differently.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unclear why the setSelectionInterval is actually any better than addAccessibleSelection. Both have a similar implementation, in some cases, the addSelectionInterval just calls setSelectionInterval, and seems that posts similar notifications. So it is unclear why the setSelectionInterval works fine and addAccessibleSelection does not work.

Copy link
Contributor Author

@savoptik savoptik Nov 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried calling clearSelection again instead of adding an interface:

                if (as == null) return;
                if (pac.getAccessibleRole().equals(AccessibleRole.LIST)) {
                    as.clearAccessibleSelection();
                }
                as.addAccessibleSelection(i);

After that I changed the call to javax.swing.JList.AccessibleJList#addAccessibleSelection

JList.this.addSelectionInterval(i, i);

on the

JList.this.getSelectionModel().setSelectionInterval(i, i);

Nothing changed. VoiceOVer still freezes.
I think that the problem is not that setting the selection doesn't work, but that clearing the selection before setting causes the VO to fris.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some probably related points, the implementation of the addAccessibleSelection for the JComboBox. is the same:

            clearAccessibleSelection();
            JComboBox.this.setSelectedIndex(i);

It is possible that it also hangs?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the problem is not that setting the selection doesn't work, but that clearing the selection before setting causes the VO to fris.

Are you sure that the selection should be always cleared in this method, how it will work if the jlist supports multiline selection? Or voice over does not support multyline selection?

As a fix, you can delete listeners from the jlist so your request to clear the current selection will not be sent back to the voiceover. Or you can add a new method to the AccessibleSelection interface with the default implementation "clearAccessibleSelection+addAccessibleSelection". Then override it in the jlist to call setSelectedIndex().

@savoptik
Copy link
Contributor Author

@mrserb
Some probably related points, the implementation of the addAccessibleSelection for the JComboBox. is the same:

        clearAccessibleSelection();
        JComboBox.this.setSelectedIndex(i);

It is possible that it also hangs?

I'm not sure if this should affect comboboxes. The combo box is arranged differently. When the ComboBox is opened, the focus is moved not to the list, but to the root panel, and from it it is proxied to the list items, which are a set of buttons that are not hierarchically or in any way connected.


Are you sure that the selection should be always cleared in this method, how it will work if the jlist supports multiline selection? Or voice over does not support multyline selection?
As a fix, you can delete listeners from the jlist so your request to clear the current selection will not be sent back to the voiceover. Or you can add a new method to the AccessibleSelection interface with the default implementation "clearAccessibleSelection+addAccessibleSelection". Then override it in the jlist to call setSelectedIndex().

As stated in the pull request description. We cannot yet implement multiple selection with VO quick navigation, as this part of the A11y API is not described by Apple.
Calling ClearSelection will really prevent you from implementing multiple selection in the future, so we propose a new interface that will allow you to implement multiple selection in the future. Because it gives access not only to different methods of setting the selection, but also to the selection mode.
The proposed solution with removing the listener will not solve the problem of binding this piece of code to the JList that we are solving.

@bridgekeeper
Copy link

bridgekeeper bot commented Dec 26, 2022

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

@mrserb Please see my comment above in which I answer your questions.

@bridgekeeper
Copy link

bridgekeeper bot commented Jan 23, 2023

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

Please do not close this pull request.

@bridgekeeper
Copy link

bridgekeeper bot commented Feb 21, 2023

@savoptik This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

@savoptik
Copy link
Contributor Author

Some comment so that the pull request is not closed.

@savoptik
Copy link
Contributor Author

We will think about solving this problem and try to offer the best.

@savoptik savoptik closed this Feb 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client client-libs-dev@openjdk.org csr Pull request needs approved CSR before integration rfr Pull request is ready for review

Development

Successfully merging this pull request may close these issues.

4 participants