Play silence to keep audio device awake

[mltony]

Link to issue number:

Fixes #14386

Summary of the issue:

Play silence in order to keep audio device open

Description of user facing changes

By default most users won't notice anything as default volume is 0 (playing silence).
I added two config options in the advanced panel: silence duration (default = 30 seconds) and white noise volume (default is set to 0). Playing white noise would be helpfulin order to debug any user issues related to audio dropout.

Description of development approach

Most of heavylifting has been done by @jcsteh.
In wasapi.cpp there has been added class SilencePlayer that spawns a new thread, that just plays silence/white noise when requested. Requests are coming from python code in nvwave.py from WasapiWavePlayer.feed() function.

Testing strategy:

Tested with white noise that it is being played as expected. Tested that config changes reflect immediately.
Couldn't test with silence as apparently my bluetooth headphones are not suffering from dropout problem in the first place.

Known issues with pull request:

N/A

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

[AppVeyorBot]

• PASS: Translation comments check.
• PASS: Unit tests.
• PASS: Lint check.
• FAIL: System tests (tags: installer NVDA). See test results for more information.
• Build (for testing PR): https://ci.appveyor.com/api/buildjobs/lo7j21c5tb9ls22r/artifacts/output/nvda_snapshot_pr16099-30837,9a8fbb01.exe
• CI timing (mins):
  INIT 0.0,
  INSTALL_START 0.9,
  INSTALL_END 0.9,
  BUILD_START 0.0,
  BUILD_END 26.1,
  TESTSETUP_START 0.0,
  TESTSETUP_END 0.4,
  TEST_START 0.0,
  TEST_END 10.5,
  FINISH_END 0.2

See test results for failed build of commit 9a8fbb01d5

[cary-rowen]

I'll test this after you fix the CI build fail

[lukaszgo1]

Please re-initialize nvwave when user restores config to saved / defaults.

[mltony]

No need to reinitialize since it reloads current settings for duration and volume automatically.

[CyrilleB79]

To me, these options seem too specific / technical for most of the users, including some users impacted by bluetooth device audio cut-off. Not every user knows what a white noise is, and from a user point of view, playing silence or with noise is not very meaningful.

Playing white noise would be helpfulin order to debug any user issues related to audio dropout.

First of all, if white noise is only for debug purpose, it should not be in audio settings IMO, maybe just in advanced settings, or better, in an add-on.
If instead white noise is required by some device or sound cards, I think that we should just enforce white noise for everybody. In any case this white noise should not be audible, so it should not make a difference with full silence.

Personally, I would see just one option "Duration of audio inactivity (in seconds) before closing the audio stream". Or even simpler: a checkbox "Keep the audio device/stream always open". Then the in the User Guide, give information to the user to help setting this value, i.e.:

• too low = audio may be cut-off more often with a Bluetooth device
• too high = the battery discharges faster.

[mltony]

@cary-rowen,
Feel free to test this now. installer NVDA test ids failing for everyone.

@CyrilleB79

these options seem too specific / technical for most of the users
it should not be in audio settings

I already put both settings in advanced panel. Most users should be happy with default values.

Some users complained to Jamie that this patch didn't fix their audio issue. We can either try to guess what happened to those users. Or alternatively we can ask them to switch to white noise and this will give us more information, because:

1. If white noise is not audible - something is really broken.
2. If white noise is audible but from another audio output device, then there is a problem with synchronization.
3. If white noise is audible from correct audio output, then something is wrong with auido
   If you take away white noise option, then there won't be a good way to debug those issues.

• too low = audio may be cut-off more often with a Bluetooth device
• too high = the battery discharges faster.

Updated doc.

[CyrilleB79]

Sorry, @mltony, I had overlooked your PR and have not checked where the options were located.

[cary-rowen]

Hi @mltony
I tested this and it worked with my bluetooth headphones, and in the past I needed to use the bluetooth audio add-on.

Thanks

[beqabeqa473]

these all should be configurable to not play silence or whitenoise, because most of the wireless headsets have a possibility to turn them all upon inactivity time.

…

On 1/28/24, Rowen ***@***.***> wrote: Hi @mltony I tested this and it worked with my bluetooth headphones, and in the past I needed to use the bluetooth audio add-on. Thanks -- Reply to this email directly or view it on GitHub: #16099 (comment) You are receiving this because you are subscribed to this thread. Message ID: ***@***.***>

-- with best regards Beqa Gozalishvili Tell: +995593454005 Email: ***@***.*** Web: https://gozaltech.org Skype: beqabeqa473 Telegram: https://t.me/gozaltech facebook: https://facebook.com/gozaltech twitter: https://twitter.com/beqabeqa473 Instagram: https://instagram.com/beqa.gozalishvili

[mltony]

@beqabeqa473

Silence duration and white noise volume are both configurable.

most of the wireless headsets have a possibility to turn them all upon inactivity time.

Not sure I understand this comment. Most Bluetooth headphones enter standby mode soon after each utterance which causes speech dropout. As far as I know there is typically no way to configure standby mode on vast majority of Bluetooth headphones.

[beqabeqa473]

No, i ment 2.4G wireless headset, they are beeing turned off after some period of inactivity which is configurable, it is 10 minutes for me, and i don't this to go away, that is why i said that this options should be configurable to turn them all together.

…

On 1/28/24, mltony ***@***.***> wrote: @beqabeqa473 Silence duration and white noise volume are both configurable. > most of the wireless headsets have a possibility to turn them all upon > inactivity time. Not sure I understand this comment. Most Bluetooth headphones enter standby mode soon after each utterance which causes speech dropout. As far as I know there is typically no way to configure standby mode on vast majority of Bluetooth headphones. -- Reply to this email directly or view it on GitHub: #16099 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

-- with best regards Beqa Gozalishvili Tell: +995593454005 Email: ***@***.*** Web: https://gozaltech.org Skype: beqabeqa473 Telegram: https://t.me/gozaltech facebook: https://facebook.com/gozaltech twitter: https://twitter.com/beqabeqa473 Instagram: https://instagram.com/beqa.gozalishvili

[seanbudd]

I believe the white noise parameter should not be settable from the GUI, as its for dev debugging only, but keeping it has a hidden setting in nvda.ini may be appropriate

[mltony]

@seanbudd, I just dropped white noise volume from settings panel.

[cary-rowen]

If the white noise volume is only used for development and debugging, it should be recorded in an appropriate doc. Otherwise, even developers will not know how to use it because it is too hidden.

[cary-rowen]

If I remember correctly, reporting progress also has a hidden interval that can be configured.
This information should be documented, such as in the "Advanced Topics" section of the user guide.

[mltony]

Renamed to "keep audio device awake" in UX as suggested. In user guide I renamed the section as well, but I left explanation that this is achieved by playing silence. Hope that's ok.

[Adriani90]

@mltony and @jcsteh thanks very much for this nice work guys.

in the user guide, I think it makes sense to replace the word utterance with "speech sequence". This is easier to understand especially for people who need easier language. @mltony what do you think?
Also I would mention that this does not affect only bluetooth devices positively, but also systems with lower performance, and virtual machines such as virtual desktops in Citrix environments.

[mltony]

@Adriani90, updated utterance -> speech sequence.
Re other use cases, what evidence do we have that silence positively affects systems with lower performance? My wife is running NVDA on a modest 8-year old laptop and she doesn't experience any problems with NVDA audio. I guess there must be users who are running NVDA on even older computers. But then I'd think just being low performance wouldn't necessarily imply bad audio quality - NVDA in general doesn't consume much resources at all. Maybe there is some old technology like a specific brand of motherboards that suffers from this problem? I don't know whether we should go into that rabbithole of investigating which old equipment causes dropout in NVDA speech, but I would prefer to have some evidence (like user testimonial) that silence indeed helps on their old equipment.
Same thing for citrix. I previously ran NVDA in VMWare Player and VirtualBox and I recall audio quality wasn't that perfect, but it didn't improve from Bluetooth Audio add-on. And the quality problems wer different: every few seconds there was some audible glitch in sound instead of beginnings of each utterance dropping, which would have been a sign of audio device being closed. My guess was that there was high latency between host and guest that caused occasional dropout of some small chunk of audio buffer. And in my understanding this own't be fixed by this PR.

[Adriani90]

At least in Citrix I am using NVDA at work and without the bluetooth addon the speech is cutting and crackling alot. I am not using any wireless headset, but a wired headfone. The bluetooth addon definitely has a positive effect on this. Without it the citrix virtual desktop is literarly unusable for me.  Von meinem iPhone gesendetAm 07.02.2024 um 23:38 schrieb mltony ***@***.***>: @Adriani90, updated utterance -> speech sequence. Re other use cases, what evidence do we have that silence positively affects systems with lower performance? My wife is running NVDA on a modest 8-year old laptop and she doesn't experience any problems with NVDA audio. I guess there must be users who are running NVDA on even older computers. But then I'd think just being low performance wouldn't necessarily imply bad audio quality - NVDA in general doesn't consume much resources at all. Maybe there is some old technology like a specific brand of motherboards that suffers from this problem? I don't know whether we should go into that rabbithole of investigating which old equipment causes dropout in NVDA speech, but I would prefer to have some evidence (like user testimonial) that silence indeed helps on their old equipment. Same thing for citrix. I previously ran NVDA in VMWare Player and VirtualBox and I recall audio quality wasn't that perfect, but it didn't improve from Bluetooth Audio add-on. And the quality problems wer different: every few seconds there was some audible glitch in sound instead of beginnings of each utterance dropping, which would have been a sign of audio device being closed. My guess was that there was high latency between host and guest that caused occasional dropout of some small chunk of audio buffer. And in my understanding this own't be fixed by this PR. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[mltony]

@Adriani90 , fair enough, I added a line about that in the user guide. Now I also vaguely remember someone on NVDA mailing list was saying their laptop's built-in audio was suffering from audio cutoffs and Bluetooth add-on indeed helped, so I mentioned that as well.

[Qchristensen]

All reads well, good work.

[jcsteh]

@mltony Thanks very much for driving this to completion. :)

[XLTechie]

I'm curious why it is thought that "speech sequence" makes sense to end users? Wouldn't something like "each spoken phrase", or describing it as "after speech stops" be more comprehensible? Even the original, "utterance", comes across as a more user friendly word, although perhaps some other languages have issues with it as @Adriani90 implied. I would rather see us describe it with as few technical terms as possible. "Speech sequence" strikes me as a term of art, which is not well defined outside our developer community.