Mute microphone command

[mltony]

Link to issue number:

Closes #16056

Summary of the issue:

A command to mute/unmute microphone.

Description of user facing changes

Added a global command to mute/unmute microphone. No default gesture assigned.
Added wasapi notification listener; when it detects microphone mute/unmute event - it speaks it. This would speak notification even if microphone is muted by other means, e.g. laptop-specific keystroke or windows settings.

Description of development approach

Made use of pycaw for both mute command and notification listener.

Testing strategy:

Tested manually on Windows 11.

Known issues with pull request:

Wasapi notification comes in with a noticeable delay of about 1 second.

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

[codeofdusk]

Added wasapi notification listener; when it detects microphone mute/unmute event - it speaks it.

Makes sense – I'm guessing this is indicated visually, and a screen reader report here gives us equivalent access.

Added a global command to mute/unmute microphone. No default gesture assigned.

This feels... out of place in a screen reader. Doesn't Windows have a hotkey to do this now?

[mltony]

@codeofdusk ,

Doesn't Windows have a hotkey to do this now?

No, see explanation in the original feature request.

[XLTechie]

How does the mute control work if you have more than one microphone, or more than one sound device with microphone inputs? I, for example, have an on-board microphone in one of my laptops. Of course it's junk, so I never use it. However, on that same laptop, I have a six-channel audio interface attached, with two XLR microphone inputs (one of which I use). What is muted if I use this control? (And yes, I know I could just try the build. But I would like to know the intended outcome first.)

[mltony]

@XLTechie, the global command would mute the default microphone. Notification will be spoken if any microphone is muted. If not default microphone is muted, it would also speak its name; while for default one it'll just say "muted/unmuted microphone".

[Brian1Gaff]

Many individual programs do, like Zoom etc. It all depends on what you term microphone here. I use line in and out as a direct loop through the sound card. However a program can still mute any sound going to zoom etc in software. Brian

…

-- ***@***.*** Sent via blueyonder.(Virgin media) Please address personal E-mail to:- ***@***.***, putting 'Brian Gaff' in the display name field. ----- Original Message ----- From: Bill Dengler To: nvaccess/nvda Cc: Subscribed Sent: Saturday, March 09, 2024 8:14 PM Subject: Re: [nvaccess/nvda] Mute microphone command (PR #16286) Added wasapi notification listener; when it detects microphone mute/unmute event - it speaks it. Makes sense – I'm guessing this is indicated visually, and a screen reader report here gives us equivalent access. Added a global command to mute/unmute microphone. No default gesture assigned. This feels... out of place in a screen reader. Doesn't Windows have a hotkey to do this now? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[bramd]

The notifications may indeed be useful, but I wonder if we are getting more information than visual users here. As far as I know, Windows shows a notification icon in the notification area/system tray when an app uses the microphone to alert the user to this, however I don't think there is a clear global visual indicator for the mic status. Would it be a better translation of visual information if we give a notification as soon as the microphone notification icon is present?

Some programs, like Zoom or Teams send their own notifications when the microphone is muted/unmuted, so with the global notification on top that might be a bit much? In other programs, an active notification might not be send and there we could enable this feature to have information that is equal to what is visible on the screen. Also, a short sound may be considered instead to not disturb an ongoing voice communication so much.

To recap, I'm not against these notifications and think they might be very useful in some circumstances, but it raises the question if we should provide more information than what is given to visual users.

[beqabeqa473]

I think it is upto software what kind of notification it subscribes to. I don't see a point in this last comment. Lots of wireless headsets have a possibility to identify in their software if microphone is muted or unmuted, and why nvda should not have such possibility? I don't think there is something bad with it.

…

On 3/11/24, Bram Duvigneau ***@***.***> wrote: The notifications may indeed be useful, but I wonder if we are getting more information than visual users here. As far as I know, Windows shows a notification icon in the notification area/system tray when an app uses the microphone to alert the user to this, however I don't think there is a clear global visual indicator for the mic status. Would it be a better translation of visual information if we give a notification as soon as the microphone notification icon is present? Some programs, like Zoom or Teams send their own notifications when the microphone is muted/unmuted, so with the global notification on top that might be a bit much? In other programs, an active notification might not be send and there we could enable this feature to have information that is equal to what is visible on the screen. Also, a short sound may be considered instead to not disturb an ongoing voice communication so much. To recap, I'm not against these notifications and think they might be very useful in some circumstances, but it raises the question if we should provide more information than what is given to visual users. -- Reply to this email directly or view it on GitHub: #16286 (comment) You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

-- with best regards Beqa Gozalishvili Tell: +995593454005 Email: ***@***.*** Web: https://gozaltech.org Skype: beqabeqa473 Telegram: https://t.me/gozaltech facebook: https://facebook.com/gozaltech twitter: https://twitter.com/beqabeqa473 Instagram: https://instagram.com/beqa.gozalishvili

[XLTechie]

@bramd wrote:

The notifications may indeed be useful, but I wonder if we are getting more information than visual users here. As far as I know, Windows shows a notification icon in the notification area/system tray when an app uses the microphone to alert the user to this, however I don't think there is a clear global visual indicator for the mic status. Would it be a better translation of visual information if we give a notification as soon as the microphone notification icon is present?

It is not always best to get precisely analogous information to what the sighted get. We have to get information that gives us comparable understanding. There is a difference. A sighted user can see that the microphone icon is visible. Seeing this does not impede the use of the microphone. But if NVDA were to indicate as soon as that icon lights up, it would probably be picked up by the microphone. That means the user will have to take steps to avoid hearing or picking up that notification. This is a case where precisely analogous information hurts accessibility, not promotes it. Whereas an indication when the microphone is muted/unmuted, will probably come before the microphone needs to be used. The user gains access to the same information, but in a way that is far more useful to a speech user, than the more accurate sight-compared notification would be.

[Adriani90]

Sighted people are also looking for a solution in this regard, see different discussions on Redit. I found a tool, that solves exactly this use case:
https://github.com/bralexc/mute-unmute-meetings

@mltony given this tool is also something sighted people use, is it something you could do too for solving your use case? Otherwise I don't see a strong argument why this feature should be part of NVDA core if there is already a tool for it. That tool even allows you to define audio indication when the Microfone is muted or unmuted.

[seanbudd]

@bramd - many keyboards (including my laptop) have a microphone mute function key with an associated light to indicate if it is toggled. This issue was initially triaged as there is usually a visual notification on the screen and sometimes keyboard lights to indicate microphone muting. I don't think we necessarily need the keyboard command to perform this, as there are other (e.g. keyboard, app specific) solutions and it is slightly out of scope, but the feature is pretty minor and very handy.

[seanbudd]

Generally looks good to me, thanks @mltony

[XLTechie]

toggle microphone mute command allows you to mute or unmute default microphone

with a single keystroke. "the" default microphone

[Adriani90]

@mltony can you make sure the original state of the microfone is returned when closing NVDA? Otherwise we run into the same problem as in #16292.

And we don't really want to make sighted users working on the same machine confused.

I still think implementing an announcement of Microfone state when changed globally might be rather something that belongs to NVDA, and not the command itself. However, the native notification of mic state should be preserved when changing with the native scripts of the coresponding app.

[AppVeyorBot]

• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/99ymmcggwtrln9la/artifacts/output/
• CI timing (mins):
  INIT 0.0,
  INSTALL_START 1.1,
  INSTALL_END 0.9,
  BUILD_START 0.0,
  FINISH_END 27.3

See test results for failed build of commit 908cb9f8e8

[CyrilleB79]

@mltony, I have tried to test this build, fixing the two syntax errors below.

Just wanted to report the following issue:
I have assigned a key (NVDA+shift+delete) to the mute / unmute command. Trying to use the command, it seems to change the mute state of the mic.
However, I cannot hear any mute / unmute notification. Is this expected?

I am running from source, under Windows 10 22H2 (AMD64) build 19045.4046.

[mltony]

@CyrilleB79 , I can reproduce this on Windows 10. Will try to investigate.

[mltony]

Ok, here are more results of my testing:

1. I have a Windows 11 laptop. Mute notifications work as expected.
2. I have a Windows 10 laptop. Mute notifications work as expected.
3. I have a virtual machine with Windows 10. It's got a microphone device. Mute notifications are not coming at all.

@CyrilleB79 , do you experience your issue on a real installation or VM?
I couldn't figure out the difference between two windows 10 installations. I verified that notification are not coming even when I use pycaw example, so the issue is not with my code, but most likely with wasapi itself.
Googling doesn't return any relevant results at all - I guess not many people use this type of audio notifications.
The conclusion I would like to make is this: wasapi notifications appear to be unreliable. Given that, do we want to still include them in the coreand risk having bug reports from people for whom they don't work? Or better not include them?
I am thinking that the best course would be to make mute mic command to speak its own update, but keep audio notifications, but make a note in user_guide saying that these notifications are not reliable.

[CyrilleB79]

My test was done on a real installation, but with no recording or application using the mic.

Same result with a second Windows 10 machine running Windows 10 2004 (AMD64) build 19041.388.
On this second machine, I have tested the mute/unmute command while audacity is recording and the notification is not heard; the mic is correctly muted/unmuted though.

In addition, I have 3 or 4 notifications heard when I open audacity, before I record anything. May it be a matter of sound card disabling the mic when not needed.

[mltony]

At this point I'm biased to drop mute notifications all together. While it works perfectly on my Windows 11 laptop, it has problems on both my Windows 10 installations and also Cyrille reported similar problems on Windows 10 as well.
So the symptoms I'm observing are:

1. Sometimes mute notification doesn't fire at all.
2. Sometimes we receive multiple mute notification. On my Windows 10 laptop manually unmuting it in settings (well unchecking "disable microphone" checkbox) triggers 37 notifications in a row.
3. Sometimes we receive mute notification on Windows login - e.g. without even explicitly muting/unmuting microphone.
4. (reported by Cyrille) Mute notifications are fired when opening Audacity.

I did some debugging, and checked notifications with pycaw example - I see the same symptoms, so likely this is not caused by a bug in my code. I don't think pycaw is at fault either - it's just a thin wrapper over com interfaces. Most likely something is wrong with Windows 10. It's possible that Microsoft fixed that only in Windows 11. Would be great if anyone else could test if this works on Windows 11. But other than that - if people don't have any better ideas - I might get rid of notifications and have mute microphone command speak microphone status.
Another potential idea is to have a thread check whetehr microphone is muted say every 1 second and announce updates there. But I'm not excited about creating a new thread.

[Adriani90]

But if it works on Win 11 it should be good enough to go.  for Win 10 it will need an external fix which means we have to take it up with Microsoft. This is not the only case which works better in Win 11 and needs an external fix on Win 10. If you are sure that the root of this problem is not in your code, then this should be fine for this case. But ofcourse NV Access has the last decision hereVon meinem iPhone gesendetAm 15.03.2024 um 22:55 schrieb mltony ***@***.***>: At this point I'm biased to drop mute notifications all together. While it works perfectly on my Windows 11 laptop, it has problems on both my Windows 10 installations and also Cyrille reported similar problems on Windows 10 as well. So the symptoms I'm observing are: Sometimes mute notification doesn't fire at all. Sometimes we receive multiple mute notification. On my Windows 10 laptop manually unmuting it in settings (well unchecking "disable microphone" checkbox) triggers 37 notifications in a row. Sometimes we receive mute notification on Windows login - e.g. without even explicitly muting/unmuting microphone. (reported by Cyrille) Mute notifications are fired when opening Audacity. I did some debugging, and checked notifications with pycaw example - I see the same symptoms, so likely this is not caused by a bug in my code. I don't think pycaw is at fault either - it's just a thin wrapper over com interfaces. Most likely something is wrong with Windows 10. It's possible that Microsoft fixed that only in Windows 11. Would be great if anyone else could test if this works on Windows 11. But other than that - if people don't have any better ideas - I might get rid of notifications and have mute microphone command speak microphone status. Another potential idea is to have a thread check whetehr microphone is muted say every 1 second and announce updates there. But I'm not excited about creating a new thread. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>

[mltony]

Makes sense; then it would be good if people can test if this works correctly on Win11.

[mltony]

More testing shows notifications are not reliable on my Windows 11 as well. My configuration is laptop with windows 11; it has two microphones: builtin microphone array, and microphone in my bluetooth headphones.

1. When Bluetooth microphone is set as default, I get all notifications correctly. From both microphones. I get no repeating notifications.
2. When builtin microphone array is set as default, then I get no notifications at all. Neither from builtin nor from bluetooth. Pycaw example listener also doesn't get any notifications in this case.

I am biased to conclude that notification API (specifically IMMNotificationClient::OnPropertyValueChanged) is broken on Windows side (also seems like in Windows 11 Microsoft tried to fix it and mostly fixed it, but still not all the way) and we shouldn't rely on it.