-
-
Notifications
You must be signed in to change notification settings - Fork 659
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Qt 5.11+ accessibility requires updates on the NVDA side #8604
Comments
CC @michaelDCurran and @jcsteh
|
@anrocqt commented on 7 Aug 2018, 16:42 CEST:
I might be missing something, but could you please provide a executable binary of this application? That will make testing a whole lot easier I think. |
@LeonarddeR: Added a binary of the sample application statically built against the current code in our 5.11 branch. |
@anrocqt commented on 7 Aug 2018, 16:42 CEST:
Strictly spoken, NVDA uses special NVDAObject types for some QT widgets that recide in the NVDAObjects.IAccessible package. These will not apply to the UIA implementation at all, as UIA normally has preference over MSAA/IAccessible.
@jcsteh: Could you elaborate on why you commented this out, according to git blame? As a side note, I noticed that NVDA is unable to detect bold and itallic text in the text browser. @anrocqt: could this implementation be incomplete on the QT side? |
These events were disabled because certain UIA providers (I think WPF 2.5 or something like that from memory) flood these events and cause NVDA to become unusable. There was an issue filed for this, but I don't know the number offhand. Because there's no way we can currently selectively blacklist certain events in specific apps at the UIA core level, we had to disable them altogether. The next release of Windows 10 includes UIA core changes to deal with bad providers like this, but that isn't going to help earlier versions of Windows.
|
just want to make sure this doesn't stall, because QT is hugely important. |
@anrocqt: Testing with JAWS 2018 revealed that the formatting changes are not detected within the text browser. The font size is reported as 0 and the bold and italic states are not reported. @jcsteh commented on 10 aug. 2018 23:12 CEST:
Do you recall whether these freezes in NVDA were occurring just because of the flood of text selection events, or because every text selection event executed an event_caret internally? |
@anrocqt: Is there anything to add now QT 5.12 has seen the light? |
Can this be revisited now? This has affected MuseScore and other Qt applications. It seems that Qt is doing everything right, but NVDA is purposefully ignoring accessibility events to compensate for bugs in other (non-Qt) applications. Here are the relevant lines in
I appreciate that the changes allowed NVDA to work with those applications, but it also removed any incentive for developers to fix bugs in those applications. Worse, if the developers test with NVDA they might come to the conclusion that there are no bugs in their application. In the past it was necessary for screen readers to go to great lengths to make non-accessible applications accessible. However, times are changing, and the onus is now on application developers to make their applications accessible. Let's reward applications that get it right rather than penalising them for the sake of preserving accessibility in applications that get it wrong. |
It's not just about allowing NVDA to work with those applications. The problem is that with these events enabled, any app based on those frameworks (whether the author cares about accessibility or not) will cause a complete hang of NVDA. Prior to Windows 10, there is no way NVDA can work around this. That said, this should only affect reporting of text selection via the mouse. Text selection via the keyboard should work, as should anything else (focus, property changes, etc.). I don't understand how this could cause the issues being experienced with MuseScore. Surely it's not using text selection events to report cursoring around the Score? |
I've compared NVDA with Narrator with regard to MuseScore, and at least with NVDA, combo boxes do not behave as they should, that is, arrowing through the combo box without opening it does not announce selection changes, whereas with Narrator, this works just fine. |
The QComboBox comboboxes consist of a root object (the combobox itself) and its children (a list with the combobox items, and an edit control). It looks like that NVDA is not receiving any PropertyChangedEvents when changing the value of the combobox without expanding it. Note that eventHandler.shouldAcceptEvent tends to drop some events, but that's certainly not the case here, as the events aren't even entering the event handler of the UIAHandler. Therefore, I have no idea what's going on here. Cc @michaelDCurran Having said that, the QComboBox neither implements a ExpandCollapse nor a Selection pattern, which, according to the Microsoft Docs, are required. Cc @anrocqt |
As @LeonarddeR said, navigation within unexpanded combo boxes is not being announced by NVDA. Interestingly, this was working under Qt 5.9 (which used MSAA) but not under Qt 5.12 (which uses UI Automation). There are other examples of this:
If you want to confirm that the above is indeed true then you will need an old version of MuseScore that uses Qt 5.9. Subsequent versions use Qt 5.12, and the MuseScore developers are reluctant to switch back. Regarding the Score View, I suspect that MuseScore was using a "trick" to get NVDA to read whatever was in the status bar, so it's possible that this is an issue at MuseScore's end. Nevertheless, the "trick" was working under Qt 5.9 and MSAA and now it is broken under Qt 5.12 and UI Automation.
If we are doing something wrong here then it would be very helpful if somebody could point us in the direction of a tutorial / example of how to implement accessibility for a custom control. Most accessibility guides just say "don't use custom controls, use built-in controls", but MuseScore's Score View is a custom control out of necessity (after all, if there was a standard way to display musical symbols in Windows then there would be no reason for MuseScore to exist). |
It would be great if somebody could try building it with the relevant lines in
I tried to build NVDA myself but I don't have enough room on my PC for all the dependencies. |
I've already done this, without luck.
|
@LeonarddeR, thanks! You didn't notice any change at all?
Cursoring changes the selection, but what event would you expect to see here? |
I'm not sure. It really depends how the score accessibility is implemented
and I've never tested it. What I do know is that it's not an editable text
control (and text infers very specific ideas about characters, words and
lines), so text selection events definitely aren't appropriate.
|
Is there a UI Automation event we can fire that would force NVDA to read the text in the status bar, or arbitrary text of our choosing that is not necessarily displayed anywhere on the screen? |
@shoogle: If I understand you correctly, all was well with older versions of MuseScore based on qt<5.11. What was the latest version that utilized this, so I can investigate the differences? |
I haven't investigated or tested this and don't have time to do so. The following is provided in the hopes that it may help, but without any guarantee of usefulness. :)
The way MSAA and UIA handle value is quite different. In MSAA, there is only one value change event and it represents a change to the accValue property, which is generally a string. In UIA, there are two concepts of value, represented by the RangeValue pattern (numeric) and Value pattern (text). It looks like Qt only fires an event for the RangeValue change, but I'm guessing that the value in this case is text. To make things more confusing, NVDA only has one concept of value. If the RangeValue pattern is present, it will choose RangeValue. Otherwise, it chooses Value. Qt exposes the RangeValue pattern if the accessible has the value interface (numeric). I don't know whether this control exposes that or not; I guess not, since it's not a numeric control. So, to fix this, I think:
As an ugly, hacky workaround, you could perhaps use the name instead of the value to expose the information. That would of course override any proper name you have set. |
Dropbox 70 switched to qt 5.12 and also suffers from the combobox issue. I'm marking this P2 to start with. Update: Also discovered that JAWS suffers from exactly the same issue with regard to combo boxes. Selection changes are only exposed when the combobox is expanded. |
Hi. Thanks for reporting the issues you found with the Qt accessibility framework on Windows. I apologize for the delayed response. About some of them:
In spite of what the docs say, it seems Microsoft is not implementing the Selection pattern in their own combo boxes. I checked with the Inspect utility, examining a combo box in the folder options dialog in File Explorer, on Windows 10, and it only implements the Value and ExpandCollapse patterns. And it is working with NVDA. Only the list view inside the combo box structure implements the Selection pattern, as does Qt. Also, the ExpandCollapse pattern makes NVDA say "Expanded"/"Collapsed", but apparently only when doing it with the mouse, which does not seem to be a concern. However, you are right about the missing UI Automation notifications when the combo box is changed with the keyboard without expanding it. Also, notifications were not being sent for updated string-type values (Value pattern), but only for range-based ones (RangeValue pattern) which probably explains the missing updates in MuseScore. I created this bug report to track the missing notifications: https://bugreports.qt.io/browse/QTBUG-75066 It should fix the combo box issues, and hopefully the Score View selection as well.
I have checked with Narrator, and it is saying both the names and descriptions, so it may be an issue in NVDA. Perhaps due to differences on how these fields are returned by MSAA and UI Automation. If you can build Qt 5.12 yourselves with the proposed patch, please let me know if it actually fixes the issues you are experiencing, as well as any other issues you may have found. Thanks, André |
As of dropbox 72 stable, they switched to QT 5.12 |
Many thanks to @jcsteh for identifying the issue with Value pattern events, and to @anrocqt for fixing them in Qt! I just tested in Qt 5.12.4 and can confirm that the issues with combo boxes and score view selections in MuseScore have disappeared.
Narrator has now stopped announcing descriptions too, at least by default. However, they are available if you press Narrator+0 (i.e. CapsLock and Zero). Does NVDA have a similar shortcut to announce descriptions? (I looked but couldn't find one.) |
cc: @feerrenrut |
@anrocqt This got a bit out of my view. How is the current state of QT accessibility? I heard rumours of severe brokenness in version 5.14. |
Now that QT is going more and more towards version 6.0, is there any status update on this issue? |
Was this issue solved by NVDA 2020.3 Beta 1 – because it wasn't closed? |
Fixed in pr #10889 |
@anrocqt If there are any remaining problems, I'd suggest opening new issues for them separately. |
Since NVDA maps FullDescription to description, it no longer announces HelpText. How do users get the message in HelpText when there are both FullDescription and HelpText? Is there a setting or shortcut to report HelpText? |
@michaelDCurran wrote in #10889 (comment):
May be we should yet consider splitting description and help text. @chigkim: Could you please open a new issue for this? |
Hi Team Note : When I tested this on Window Inbuilt Screen reader like narrator it read properly and also JAWS also read same behavior as Narrator screen reader . Please support if possible .What we need to do ? |
Hello.
I can confirm that NVDA has several issues if we are talking about qt framework.
Even narrator works much better in this area.
…On 10/11/22, Aksh Singh ***@***.***> wrote:
Hi Team
I am facing one issue with NVDA Screen Reader when the focus goes to the
Listview in Qt /QML Application
it is not reading 1 of 5 ,Let's total 5 item in the list and current item is
1 . then it should read name of the item and then read selected and then 1
of 5 or 1 of 5 item selected .
Note : When I tested this on Window Inbuilt Screen reader like narrator it
read properly and also JAWS also read same behavior as Narrator screen
reader .
Please support if possible .What we need to do ?
--
Reply to this email directly or view it on GitHub:
#8604 (comment)
You are receiving this because you are subscribed to this thread.
Message ID: ***@***.***>
--
with best regards Beqa Gozalishvili
Tell: +995593454005
Email: ***@***.***
Web: https://gozaltech.org
Skype: beqabeqa473
Telegram: https://t.me/gozaltech
facebook: https://facebook.com/gozaltech
twitter: https://twitter.com/beqabeqa473
Instagram: https://instagram.com/beqa.gozalishvili
|
Could you please provide some examples to test with? |
Hi leonardder , |
Could we discuss new problems in new issues please? I know it's tempting to post here because a Qt developer is subscribed, but the risk is that they will unsubscribe if all the comments are about NVDA bugs that happen to affect Qt software, rather than things that are actually the fault of Qt. If you create a new issue and it does turn out to be on the Qt side rather than the NVDA side, you could leave a single comment in this thread linking to it to let subscribers know that it exists, but could we avoid having the discussion here please? |
@shoogle I'm pretty sure that these problems are a combination of NVDA and QT. Both need to do their part. QT needs far more attention from NVDA if this problem is to be resolved. |
|
@shoogle Your first argument is correct, or would be, if it weren't for the fact that Orca, on Linux, has similar problems regarding QT accessibility in certain contexts. A standard |
@ethindp, what do QLineEdits have to do with the ListView problem raised earlier? You seem to be talking about something else entirely, which further illustrates the need to use separate issues to avoid confusing people. Qt provides an abstraction layer for the underlying accessibility frameworks, which are very different on Windows and Linux. If Orca and NVDA exhibit similar problems then that could indicate a Qt problem, but not when all the other Windows-based screen readers are working properly with Qt, which appears to be true in the case of the ListView problem.
Yes please! I don't mind if this is used as the central issue, but I don't think the back-and-forth discussion about whether a particular problem is or is not on the Qt side should take place here. |
Hi @shoogle , I am stuck In between that's why I post here this issue . I also faced one another problem with NVDA . NVDA also does not properly work on slider . Only First time it read properly whenever we move the slider using the --> and <-- (left or right button) it will not speak anything while other screen reader like narrator read slider at 5 (current value of slider). |
@ethindp and @cppqtdev as far as I understand two issues are still open:
If this is correct, could you please open an issue with these two problems and attach a test application where we can reproduce the list and the slide issue? Preferably as a binary application or something that can be run without having to install the whole development environment. That would be very appreciated. |
We didn't get any updates from people who could reproduce this since more than two years, even though we requested to open new issues for the remaining problems including test applications to test with. |
Qt 5.11 replaced its legacy MSAA-based Windows accessibility backend with a new implementation based on UI Automation. This seems to be causing some issues with NVDA. I'm currently investigating the following bug report, which includes a detailed description of the issues, as well as a sample project to reproduce them:
https://bugreports.qt.io/browse/QTBUG-69472
While some of the issues may result from bugs in our implementation, some others may require updates in the NVDA side, which uses a custom app module for Qt applications, which may no longer apply since Qt 5.11.
I took a look at NVDA sources and saw that it supports MSAA and UI Automation, and has app modules that add custom behavior and fixes for many applications and frameworks, including Qt. The Qt app module in NVDA is based on MSAA/IAccessible, used in previous Qt versions. When used with a Qt 5.11 application, NVDA is recognizing that it supports UI Automation and using its UI Automation module.
However, this module seems to be lacking some functionality. For instance, the automatic detection of changes in text selection seems to have been disabled for some reason. The Qt framework is raising "UIA_Text_TextSelectionChangedEventId" notifications to inform UIA clients of changes in selection. However, NVDA seems to be purposely ignoring it (the mapping from UIA_Text_TextSelectionChangedEventId to the "caret" event is commented out in the _UIAHandler.py source). Also, NVDA's UIA module uses the "EditableTextWithoutAutoSelectDetection" class, instead of the "EditableTextWithAutoSelectDetection" used on the IAccessible module. I tried enabling the UIA_Text_TextSelectionChangedEventId mapping and changing the UIA module to use EditableTextWithAutoSelectDetection, and some of the compatibility issues reported, like NVDA failing to speak the mouse selection, or the lack of updates in the Braille display where fixed. However, I'm not familiar with the NVDA code, and that change will likely break other stuff.
I would suggest keeping the current custom app module in place for supporting Qt versions prior to 5.11 and using the UIA module for >= 5.11 (i.e. whenever UIA support is detected), while enabling the necessary features to make it work properly. I can help with changes on the Qt side.
Also, please instead of creating a new custom app module for Qt 5.11+ to work around bugs in our implementation or behavior unexpected for a proper UI Automation server, please report them to us so we can have them fixed on our side instead (you can assign bug reports to me).
Note: the new accessibility backend is a pure UI Automation implementation. Legacy MSAA clients are supported through the UIA-to-MSAA bridge provided by the MS UIA framework in the middle. However, the use of the bridge was broken in the initial 5.11 release, but will be fixed in 5.11.2 (the fix is already in our 5.11 branch, if you build Qt yourself). However, the use through the bridge does not provide all functionality that may be required. In particular, it does not provide IAccessible2 support, only IAccessible. Some features like caret movement detection are in IAccessible2, so I think a proper UI Automation interfacing is the way to go, instead of trying to use it in MSAA mode (also, even going through this legacy mode would also require changes in NVDA, as it detects the UI Automation support in Qt 5.11 and seems to ignore the MSAA-based app module).
The text was updated successfully, but these errors were encountered: