Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make the audio device still active for better response #5172

Closed
nvaccessAuto opened this issue Jun 20, 2015 · 17 comments · Fixed by #11024 or #11531
Closed

make the audio device still active for better response #5172

nvaccessAuto opened this issue Jun 20, 2015 · 17 comments · Fixed by #11024 or #11531
Labels
component/audio NVDA's audio output (nvWave, issues with usb audio etc). enhancement
Milestone

Comments

@nvaccessAuto
Copy link

nvaccessAuto commented Jun 20, 2015

Reported by ondrosik on 2015-06-20 21:39
Some audio devices have power management system which disables the device when it doesn't play any audio. This makes NVDA less responsive. When you press a key and NVDA wants to say something, you need to wait few miliseconds because the device needs to initialize, and after that play the sound, in this case speech synthesys. This also applies when you for example spell the statusbar or titlebar - spaces between characters are longer. The solution is really simple, Play silence. to the device all the time or somehow make it still active. When you run audio player such as winamp, NVDA is really more responsive, also when you pause the player. I think that we should integrate to NVDA something which will make the soundcard active all the time when NVDA is running. In most situations user can not disable this power management. In older soundcards, there was a comfig file or specific driver related dialog where this was possible, but nowadays it seems that mostly user can't do nothing about it.

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jun 20, 2015

Comment 2 by nvdakor on 2015-06-20 21:48
Hi,
My main concern is battery life. Although this may not matter for desktops, we may get complaints from users in the future for battery drain issues on laptops, especially when wireless headsets and headphones are used. Although performance matters, in today's mobile computing environment, battery life matters as well, especially when NVDA will be used by professionals on the move.
Thanks.

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jun 20, 2015

Comment 3 by ondrosik on 2015-06-20 22:18
OK, I agree, maybe we should make it as an option and disabled by default.

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jun 22, 2015

Comment 4 by jteh on 2015-06-22 01:31
NVDA actually already unintentionally keeps the audio device always open due to a deficiency in our tone generation code. The only exception to this is if the audio device is changed while NVDA is running. So, if you start NVDA and don't change audio devices, the device should already be kept open.

If this isn't fixing the delay issue, keeping the device open isn't going to be enough. This is rather odd because this is exactly what media players do when you pause them and you're suggesting that pausing a media player is enough.

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jun 22, 2015

Comment 5 by ondrosik on 2015-06-22 07:13
Well, I didn't notice this but you are right. when I plug my usb soundcard and restart NVDA the problem is gone. but after switching the audio device it starts responsing slovly.

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jun 22, 2015

Comment 6 by ondrosik on 2015-06-22 07:57
So maybe we need make it happen also when you switch the device.

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jul 28, 2015

Comment 7 by driemer.riemer@... on 2015-07-28 00:23
How does nvda keep the device open? I can't find a place in the code that intentionally does this. If I were going to have nvda open the device on device reloading is there a part of the code where nvda reloads the device? When I looked through nvwave, it seemed that it was defaulting to close the device. def init(self, channels, samplesPerSec, bitsPerSample, outputDevice=WAVE_MAPPER, closeWhenIdle=True):

The evidence for my thought is closeWhenIdle=True would close the device.
Thanks,
Derek

@nvaccessAuto
Copy link
Author

nvaccessAuto commented Jul 28, 2015

Comment 8 by jteh (in reply to comment 7) on 2015-07-28 01:03
Replying to driemer.riemer@…:

How does nvda keep the device open? I can't find a place in the code that intentionally does this.

It's not intentional. It's a bug/deficiency. See #400.

@bhavyashah
Copy link

bhavyashah commented Aug 19, 2017

@derekriemer I believe you were working on a Play Silence add-on at one point in time. Was that related to this ticket in any way?

@derekriemer
Copy link
Collaborator

derekriemer commented Aug 20, 2017

It seems that this is becoming more and more common with modern dell sound cards, and the unspoken addon by @camlorn randomly fixes the problem, although I don't know what he does differently than NVDA to hold the sound card hostage. @feerrenrut Any idea on priority? This issue is very annoying for those who have it, making NVDA very very annoying to use and nearly useless on new dell sound cards.Instead of hearing hello, you'd hear ello, (To be clear, the first sylibal of every utterance is cut off after a pause of speech over half a second or so).

@feerrenrut
Copy link
Member

feerrenrut commented Aug 21, 2017

Based on the impact (skipped sounds) of this, I would consider this is fairly important. The questions that come to mind are:

  • How many users does this actually impact?
  • Is a developer able to reproduce this problem?
  • What are the concrete steps to reproduce the problem?

Without being able to reproduce this, it will be difficult to fix. Essentially requiring a process of sending builds to users who report the problem until we have the confidence that this does fix the problem without introducing any other issues.

@Brian1Gaff
Copy link

Brian1Gaff commented Aug 21, 2017

@ahicks92
Copy link
Contributor

ahicks92 commented Aug 22, 2017

I have seen something like this in some form on almost every computer I have ever used any screen reader on for the last 8 or so years, fortunately (but not always) fixable by disabling enhancements in control panel.

Unspoken fixes it because Unspoken never closes the audio device and is always writing zeros. The sound card can't distinguish between that and actual audio, and so holds onto all resources. there's nothing else special in Libaudioverse that would change anything--it's just using NVWave in that add-on, not my (admittedly buggy) backends.

I'm not going to be fast to respond to e-mail for a while so if someone wants to ask specific questions it might be best to grab me via IRC or something.

@jcsteh
Copy link
Contributor

jcsteh commented Nov 2, 2017

I'm seeing this with Realtek High Definition Audio on a Lenovo P51, even with enhancements disabled. The first sound played after audio has been closed incurs about a half second of lag when a headset is connected (not USB). Curiously, the lag is less (though still not instant) when using the laptop's speakers. This happens even if I exit NVDA and cause a Windows sound to play; e.g. by pressing backspace on a button in the Run dialog. So, this is definitely something related to the audio hardware/driver. Note that updating drivers, etc. does not resolve the issue.

There are two options here:

  1. Keep the device open forever. I don't know if we want to do this because it could have implications for power. Note that we already do this in some cases anyway due to tones audio, so I guess this wouldn't be a regression strictly speaking.
  2. Close the device after a certain time of inactivity; e.g. 10 seconds. The downside to this is that when the user interacts again after this time lapses, they will incur the lag. That isn't so bad if the user is not likely to pause for that long during continual interaction.

There's also the question of whether we want to make this configurable and what the default should be. One concern is that an option like this isn't going to make sense to most users, so it should be considered an advanced setting. It's perhaps worth rolling this out as a hidden config option first so people can try it out.

@Brian1Gaff
Copy link

Brian1Gaff commented Nov 2, 2017

@Adriani90
Copy link
Collaborator

Adriani90 commented Jul 28, 2018

@jcsteh, @feerrenrut, @michaelDCurran, I have found a very helpful vbs script which solves this problem, see the issue referenced by @leonardder above. Could something like that be implemented in NVDA at start?

@michaelDCurran michaelDCurran added the component/audio NVDA's audio output (nvWave, issues with usb audio etc). label Jan 26, 2019
@Adriani90
Copy link
Collaborator

Adriani90 commented Jan 27, 2019

cc: @mltony

@mltony
Copy link
Contributor

mltony commented Jan 27, 2019

My NVDA add-on BluetoothAudio fixes this issue by playing silence to prevent wireless headsets from entering standby mode. To optimize battery usage, it only plays silence for 1 minute (configurable parameter) after the last speech command.
https://github.com/mltony/nvda-bluetooth-audio/

feerrenrut pushed a commit that referenced this issue Aug 10, 2020
…io played for 10 seconds. (PR #11024)

Fixes #5172
Fixes #10721

## Summary of the issue:

- As described in #5172, some audio drivers/hardware take a long time to open the device and/or truncate the start/end of the audio.
- As described in #10721 (comment), calling WaveOutOpen in the OneCore synth callback mysteriously blocks (and thus lags) for ~100 ms. Because we close the audio device on idle, we can trigger this problem. Although #11023 mostly fixes this, it's impossible (or at least very difficult) to resolve this completely from within the OneCore driver.
- Aside from all of this, closing and opening the audio device for rapid short utterances (e.g. rapid movement with the cursor keys or typing) doesn't seem particularly optimal. It's difficult to measure this, but I'd say mitigating this is likely to make audio performance faster/smoother.

## Description of how this pull request fixes the issue:

When WavePlayer.idle() is called and closeWhenIdle is set, instead of closing the device immediately after the audio finishes, set a timer. If audio is played before the timer elapses, stop the timer. Close the device when the timer expires.
@nvaccessAuto nvaccessAuto modified the milestone: 2020.3 Aug 10, 2020
feerrenrut added a commit that referenced this issue Sep 1, 2020
Fixes #5172
Fixes #10721
Fixes #11482
Fixes #11490

### History
#5172
Some audio drivers/hardware take a long time to open the device and/or truncate the start/end of the audio.

#10721
Calling WaveOutOpen in the OneCore synth callback mysteriously blocks (and thus lags) for ~100 ms. Because we close the audio device on idle, we can trigger this problem. Although PR #11023 mostly fixed this, it's impossible (or at least very difficult) to resolve this completely from within the OneCore driver.

PR #11024 attempted to fix these issues by waiting 10 seconds before closing the audio device"

### Problem to solve
Subsequent to #11024, there are occasional exceptions from nvwave, particularly when switching synthesisers. In particular the following cases need to be handled smoothly:
- When using Microsoft Sound Mapper, NVDA should use the Windows default device (even if it changes)
- When the NVDA configured devices becomes invalid, nvWave should fall back to Microsoft Sound Mapper
- When the NVDA configured device becomes available again, NVDA should switch back to using it.
- Handle no audio device at all.

Since these issues needed to be fixed, and also:
- Closing and opening the audio device was originally introduced to support Remote Desktop audio, this is now better served by other solutions. 
- Performance is improved by keeping it open, using a timeout means that the lag is more rare, but will still occur.

### How it is solved
Instead, now don't close the device at all. As per the discussion:
#11505 (comment)

To fix issues with current / preferred device:
- nvWave now saves the preferred device when it is constructed.
- If using a device fails, it is considered unavailable, nvWave attempts to fall back to "Microsoft Sound Mapper".
- From my testing, using "Microsoft Sound Mapper" correctly handled changing the default device (Win 10 2004). It would be helpful if others could confirm, especially on different OS versions.
- During `_idleUnbuffered`
  - The current device is checked to see if it matches the preferred device.
  - The available devices are polled to see if the preferred device is available, if so it switches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/audio NVDA's audio output (nvWave, issues with usb audio etc). enhancement
Projects
None yet
10 participants