Zoom Does Not work As Expected with NVDA

[govsta1981]

Steps to reproduce:

1. Go to this link and install the zoom client: https://zoom.us/download
2. Install the program.
3. When you launch the program, tab to either "start a meeting with video", or "start a meeting without video". Press enter on either of these buttons.
4. Press alt+a to mute or un-mute your audio. Press alt+v to turn your video on or off.
5. Press alt+h to open the chat panel.You are immediately put into the edit field to type text. Type a few words. use your arrow keys to move around the text, mostly the left and right arrow.

Expected behavior:

For step 4 above, you should hear the fact that audio is either muted or un-muted, or that video is started or stopped. For step 5 above, you should be able to navigate the text with arrow keys after typing it. It should also be noted that Zoom gives notifications of everything that goes on in the room. For example, when recording is stopped or started, what someone types in the chat panel, when screen sharing is started or stopped, and so on.

Actual behavior:

When I do the above steps, I get no notification of anything having occurred. I receive nothing telling me that an action has been taken. I cannot see text that I type into any edit field within Zoom by using arrow keys. I can, however, hear the text when I tab away from and then come back to the edit field.

System configuration:
Dual-core 2.16 ghz Intel Celeron, 4 gb ram, 500 gb hard drive. Video card is on-board Intel video.
NVDA version:
NVDA next-14601,e62a168e

NVDA Installed or portable:
Installed

Other information:

Example: Running in a VM

Windows version:

Windows 6.3 build 9600

Name and version of other software in use when reproducing the issue:

The absolutely latest version of Zoom

Other questions:

Does the issue still occur after restarting your PC?
Yes
Have you tried any other versions of NVDA?

No
Please list them and the result
Roommate uses version 2017.3, and he has same issues that I described above.

[ehollig]

I can confirm this using NVDA master-14625,b6c4d81f. The chat field has dev info of:
name: u'Input chat text Type message here\u2026'
role: ROLE_EDITABLETEXT
states: STATE_FOCUSED
isFocusable: False
hasFocus: True
Python object: <NVDAObjects.IAccessible.IAccessible object at 0x0551A950>
Python class mro: (<class 'NVDAObjects.IAccessible.IAccessible'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <type 'object'>)
description: None
location: (1045, 662, 287, 58)
value: None
appModule: <'appModuleHandler' (appName u'zoom', process ID 26136) at address 519ef90>
appModule.productName: u'Zoom Meetings'
appModule.productVersion: u'4,0,36525,0619'
TextInfo: <class 'NVDAObjects.NVDAObjectTextInfo'>
windowHandle: 2622586
windowClassName: u'ZPConfChatWndClass'
windowControlID: 0
windowStyle: 1442840576
windowThreadID: 39736
windowText: u'Zoom Group Chat'
displayText: u'Type message here\u2026'
IAccessibleObject: <POINTER(IAccessible) ptr=0x82b5e10 at 55e8f80>
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=2622586, objectID=156045680, childID=0
IAccessible accName: u'Input chat text Type message here\u2026'
IAccessible accRole: ROLE_SYSTEM_TEXT
IAccessible accState: STATE_SYSTEM_FOCUSED, STATE_SYSTEM_VALID (4)
IAccessible accDescription: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))
IAccessible accValue: exception: (-2147467263, 'Not implemented', (None, None, None, 0, None))

[feerrenrut]

Looks like this is something that could be improved, we will gladly accept a pull request to fix this.

[govsta1981]

I got myself a new computer, and it has Windows 10. Narrator reads notifications in Zoom. Not sure about edit fields.

[richardweiym]

@govsta1981 @feerrenrut @ehollig I am the Engineer from ZOOM, sometimes we need proactively announce some text in the meeting, something like "your audio is muted" "xxx started screen share", here are the method we used to realize this feature: NotifyWinEvent(EVENT_SYSTEM_ALERT, m_hWnd, (LPARAM)pAccControl, CHILDID_SELF); pAccControl is an control that implemented IAccessible interface and his role type is ROLE_SYSTEM_ALERT. it works on both JAWS and Narrator, but can't work on NVDA, we will really appreciate if you guys can also support it.

[josephsl]

Hi, NVDA does support UIA-based system alert event. I suggest using either a live region changed event or a name changed event so someone who plans to write an app module or a support class for this can implement this easily. Thanks. From: richardweiym [mailto:notifications@github.com] Sent: Friday, February 23, 2018 12:26 AM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754) @govsta1981 <https://github.com/govsta1981> @feerrenrut <https://github.com/feerrenrut> @ehollig <https://github.com/ehollig> I am the Engineer from ZOOM, sometimes we need proactively announce some text in the meeting, something like "your audio is muted" "xxx started screen share", here are the method we used to realize this feature: NotifyWinEvent(EVENT_SYSTEM_ALERT, m_hWnd, (LPARAM)pAccControl, CHILDID_SELF); pAccControl is an control that implemented IAccessible interface and his role type is ROLE_SYSTEM_ALERT. it works on both JAWS and Narrator, but can't work on NVDA, we will really appreciate if you guys can also support it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7754 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkB-ryNVBPe1Rwvyckn1vcs6hcMq5ks5tXnYrgaJpZM4QcBrX> .

[richardweiym]

@josephsl , it is not UIA-based, it is just legacy IAccessible solution...

[josephsl]

Hi, I see. As I said, because NVDA only supports UIA-based system alert, the best case for now is using generic IAccessible name change or live region event. If this is a standalone app, maybe it might be possible to let NVDA respond to an alert event. If it’s a web-based solution, perhaps a live region could work. Thanks. From: richardweiym [mailto:notifications@github.com] Sent: Friday, February 23, 2018 12:34 AM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com> Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754) @josephsl <https://github.com/josephsl> , it is not UIA-based, it is just legacy IAccessible solution... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7754 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkDPXWzZ6--xavNK7aQMKi4oa2AwIks5tXngDgaJpZM4QcBrX> .

[michaelDCurran]

I think this needs a little more investigation before we can say we won't do it. Thanks for the report. We will look into it soon.

[richardweiym]

@josephsl Thanks for you advice, it is standalone Windows Desktop Application. As we tested, "name change event" can't lead Screen Reader to announce the new text, so we uses this solution..

@michaelDCurran Thanks

[derekriemer]

@richardweiym Another thing I noticed is that some checkboxes don't announce their state change. Are you guys notifying WinEvent?

[richardweiym]

@derekriemer thanks for your remind, we will fix it

[derekriemer]

I am about to investigate EVENT_SYSTEM_ALERT.
I'll let you know if I can get NVDA to notify these.

[derekriemer]

Okay, I know how to make this work. After a few more possible flood mitigations are worked out, I'll open a pull request, and NVDA will work better with zoom's alerts.

[richardweiym]

@derekriemer thank you so much, looking forward to your fix release.

[derekriemer]

@govsta1981 @richardweiym @ehollig can you guys please test this try build to see if it solves the problem? To test this, please run NVDA, accept the license agreement, then hit continue running. This will not modify NVDA on your system in any way, shape or form if you hit continue running.

[richardweiym]

@derekriemer , Seem it still can't work, could you have a try of Zoom Client for Meetings (https://www.zoom.us/download#client_4meeting), it is free. when I press tab to navigate and I fire other event in meeting, nothing announced.

[derekriemer]

I do the following:
Run the try build.
Launch a meeting from the taskbar icon in zoom.
Then, tab to mute, and press it, or open chat pannel and type "frog" and press enter. Alerts seem to be firing.

[derekriemer]

Sorry, I typed that right before dinner. I was curious if your steps differ from mine? are you by chance using a development version where something may have changed, in which case, we'll need to incorperate the changes?
Cheers,
Derek

[richardweiym]

@derekriemer it works, thanks for your help.

[ehollig]

@derekriemer, this try build works and reports event changes like announcement of mute/un-mute audio, turn video on/off, and announcement of new chat messages. However, when pressing alt+h to type a message, using the arrow keys to review the message that you have typed still does not work as expected.

[derekriemer]

I cannot fix that, and the PR isn't supposed to fix that. If @richardweiym needs any help fixing the typing, or has a fix and can verify a fix is needed for NVDA, I'm more than happy to help.
@richardweiym thanks for the technical details of what NVDA was doing wrong, it made it much easier to fix. The main problem with chat notifications was because the chat window is a background window (The alert fires in a window which isn't a child of the current system foreground window), and so NVDA filters out the event as coming from a background task, so the user isn't alerted unnecessarily. Given the importance of zoom messages, I decided to allow them for zoom, even if zoom is not in the foreground.

[richardweiym]

@derekriemer I really appreciate for your support of Zoom!

[yangjian123]

@derekriemer thanks for supply solution for zoom's accessibility, could you please tell me when nvda's new version will be deployed, I can't wait to have a try for this feature.

[govsta1981]

The work done for Zoom can be found in [2018.3 Beta 3](https://ci.appveyor.com/project/nvaccess/nvda/build/2018.3beta3). Unfortunately, I do not have the direct link to snapshots.

[yangjian123]

@govsta1981 when i click this link, it only shows to me that 'Build not found or access denied'. where can i found the correct package. thanks

[ehollig]

Zoom has updated. There is now quite a bit of a delay between when the keystroke to mute/unute is pressed, and when it announces the status of the mute button.
IO - speech.speak (11:30:43.605):
Speaking [u'video now stopped alert']
IO - inputCore.InputManager.executeGesture (11:30:47.979):
Input: kb(laptop):alt+a
IO - speech.speak (11:30:48.519):
Speaking [u'audio now unmuted alert']
IO - inputCore.InputManager.executeGesture (11:30:49.328):
Input: kb(laptop):alt+a
IO - speech.speak (11:30:53.516):
Speaking [u'You are using the computer audio alert']
IO - inputCore.InputManager.executeGesture (11:30:56.678):
Input: kb(laptop):alt+a
IO - speech.speak (11:30:58.515):
Speaking [u'audio now muted alert']
IO - inputCore.InputManager.executeGesture (11:31:02.042):
Input: kb(laptop):alt+a
IO - speech.speak (11:31:03.516):
Speaking [u'audio now unmuted alert']
IO - inputCore.InputManager.executeGesture (11:31:06.641):
Input: kb(laptop):alt+a
IO - speech.speak (11:31:08.516):
Speaking [u'audio now muted alert']

[richardweiym]

@derekriemer @ehollig @govsta1981 Could you confirm this fix is released to product line? I don't find your fix in this package, https://www.nvaccess.org/download/

[govsta1981]

It works as expected on latest alpha, and NVDA 2018.4. From: richardweiym <notifications@github.com> Sent: Thursday, February 28, 2019 1:26 AM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Governor Staten <govsta@gmail.com>; Mention <mention@noreply.github.com> Subject: Re: [nvaccess/nvda] Zoom Does Not work As Expected with NVDA (#7754) @derekriemer <https://github.com/derekriemer> @ehollig <https://github.com/ehollig> @govsta1981 <https://github.com/govsta1981> Could you confirm this fix is released to product line? I don't find your fix in this package, https://www.nvaccess.org/download/ — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7754 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/APFDMc7WlP-ssfN0EdP-I42CP7agKcimks5vR3Z1gaJpZM4QcBrX> . <https://github.com/notifications/beacon/APFDMWs1rfGMPDkTmnZzASKPEpSnqAa_ks5vR3Z1gaJpZM4QcBrX.gif>

[richardweiym]

@govsta1981 yes, it works, thanks for your quick response.

[derekriemer]

The appModule isn't placed in %appdata%\nvda\appModules, but rather bundled in NVDA under the programFiles directory. It's there, as confirmed below.

appModuleHandler.doesAppModuleExist("zoom") returns True