-
-
Notifications
You must be signed in to change notification settings - Fork 3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Accessibility API #5836
Comments
Hello. I unfortunately do not know the exact specifics of the Apple's accessibility APIs, however as a visually impaired developer myself, my guess at the at least somewhat common accessibility related properties is:
As a basic inspiration you could look at the Firefox's accessibility tree, they are using (or at least it seems) a cross platform internal tree at least under Windows and Linux. And yes, it will have to be a tree, every accessibility API i know of expects an object tree. |
Thanks for this info, so i guess as a first step, There are some things that kivy can't know for sure unless the widgets provide the info, for example, current visibillity of a widget is not easy to compute, as some widgets can cover the screen, while being totally invisible to the user, and not preventing interactions to widgets behind it, so a widget should declare its "visible" zone in some way, so kivy can compute its impact on widgets behind it. It's easy for Button/checkbox/Popup and others, but the more specific cases will require custom code (Image is a difficult one, it could be fully transparent and won't prevent any interaction by default, but the screen reader should certainly know there is something there, i assume each "image" use in the program would require an "alt" tag, and the screen reader would simply ignore these with an empty value). In case this whole traverse/update state dance is costly and slows down existing programs, we could have a kivy setting to enable/disable the whole thing, but that would certainly reduce the number of people checking their widgets have good accessibility. |
Someting along the lines of it, yes. Also, in reaction to a widget event (e. g. state/property change), the appropriate accessibility events should be raised so a screen reader can notify the user in real time. |
And, you probably should not think of a screen reader interactions in terms of mouse actions, just to be on the same page. |
How should we represent the widgets' boundaries in the tree? (pos, size) might be sufficient for most widgets, but if it's a Scatter or something like that, we might need an arbitrary polygon. Can screen readers handle those? |
Ah -- perhaps tyrylu meant that we shouldn't represent boundaries within this tree at all, instead implementing some other API that takes an element in that tree and changes its state, without touch events being involved at all. Then, if the user is in fact using a pointing device, we can represent what it's doing by changing the state of a widget to indicate that the mouse is above it. But some users will be navigating with arrow keys and Tab and such. This API would need to either simulate touch with those, or add tab-browsing functionality to all the widgets. |
Yes, correct. |
I haven't had a chance to read this carefully, but I think we can maybe
take inspiration from `FocusBehavior`. That provides a mechanism for
widgets to opt in to the behavior as well as a method to iterate those
opted-in widgets.
For this problem, by inheriting from the accessibility behavior, the dev
could provide more info about the widget, other than the visual
representation. Such as whether it's visible, a description, a way to
increase zoom or change font or whatever we come up with as common things
that the widget should support. Then we can have a global module that when
activated walks this behavior tree, like tab does for `FocusBehavior` and
interacts with it.
But we would need to know the kind of things that such widgets need to
support.
…On Sat, Sep 8, 2018, 6:00 PM Lukáš Tyrychtr ***@***.***> wrote:
Yes, correct.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#5836 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/ABkW_vlXzRf3bSPzboXDDCrmmho5J6Ivks5uZD30gaJpZM4U-WCd>
.
|
For the supported operations, you could look at IAccessible2: http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/ |
I'm not sure whether it's better to support such accessibility as we can without |
Well, if you make such thing optional, you can be sure that many developers will just ignore it - time/management priorities, not thinking that it might be useful etc. |
OK, so the accessibility module should still work when the app does not support tabbing. I guess that means it needs to detect when a widget is being hovered over by, like, checking Some widgets already give us the information we need for operations like IAccessible2. Offhand:
It's already basically possible to use apps designed for mice without a mouse using the |
Would it help the project if I prototyped a module that made Kivy accessible on Windows using UI Automation, using whatever information is already available on the widget objects? This would be single-platform and hacky, but I think it would help us get a better idea of what changes need to be made in the widgets to make this work well. |
I can't speak for the people who need this feature, but what you suggests seems like a good fit for garden. |
Is there any further info on this work? As a blind developer I would love love love to get this moving and contribute however I can. I want to develop android apps and really want to use Kivy but the lack of accessibility is what's stopping me. Using it on windows would be nice too -- but at least there I have alternatives to Kivy. |
There is a proposal here about a API that could potentially be used for Accessibility that could use some insight from people familiar with the topic. Whether an Accessibility API would work as described in the proposal: #6724 (comment). |
I am also interested in this, and would be happy to throw in some work if that made this progress faster. Let me know if there is anything I can contribute. |
It looks like there were plans to make Kivy more accessible in the past I have no idea what recent conversations have been, but I thought the above link was relevant to this thread |
I took a quick look around and it seems implementing accessibility is actually quite complex and would require proper design with an API. There are multiple frameworks (On Windows MSAA and IAccessible, with the newest API Microsoft UI Automation; on linux IAccessible2 is used, but ATK/AT-SPI seems the more common API; on Mac there's NSAccessibility; not sure what iOS or android do).
All this shows that each platform has their own interface and if we need cross platform support we'd need to have some kind of intermediate interface. And even if we only implement one interface, it'd still need to be properly implemented using a client/server architecture to communicate with the screen readers etc. So it seems like it would take a significant amount of resources to implement this. Although perhaps just getting started may not be as hard. |
It's not clear when it'll be ready, but we may be able to use AccessKit for our accessibility API. |
Looks like AccessKit is ready! Here's the Python package for it. |
Let's move to #8547 |
There's an issue #2820 to add screenreader/UI Automation support to Kivy, but it got closed because no one was up to it.
Discussion on IRC suggests that what's needed to get started is an API for widgets to implement that can then be provided by whatever the target platform does for screenreading. That would split the work up so we can work on one platform at a time and eventually support them all.
To design that API, we need someone knowledgeable about how screenreaders work on all target platforms to decide on a subset of functionality that they all share, and which all Kivy widgets ought to support.
The text was updated successfully, but these errors were encountered: