Speech is muted after reconnecting to a Remote Desktop session if using WASAPI

[WestonThayer]

Steps to reproduce:

1. Connect to a remote PC using Microsoft Remote Desktop (remote desktop protocol/RDP)
2. Start NVDA, observe speech is heard
3. Close the remote desktop session without exiting NVDA
4. Re-connect to the remote desktop session and send some NVDA commands

Actual behavior:

NVDA speech is muted. Sounds, like switching between browse & focus modes, work.

Expected behavior:

NVDA speech is not muted, as in NVDA 2023.2 and earlier (before WASAPI was the default).

NVDA logs, crash dumps and other attachments:

log.txt

I'm using the out-of-the-box settings, so Windows OneCore voice.

You can fix audio by:

• Quitting an re-opening NVDA
• Going to Settings > Audio > Audio Output Device toggling between "Microsoft Sound Mapper" and "Remote Audio", then Apply

This issue does not reproduce if you disabled WASAPI via Settings > Advanced > Audio > Use WASAPI for audio output (requires restart) Disabled.

System configuration

NVDA installed/portable/running from source:

Repro's on installed and portable.

NVDA version:

2023.3 and later

Windows version:

Reproduces on 10 and 11.

Name and version of other software in use when reproducing the issue:

N/A, these are fresh installs of Windows and NVDA.

Other information about your system:

Using the Microsoft Remote Desktop app, but reproduces in others. I am using VMs, haven't tested against a physical PC.

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

Yes, repro's in 2023.3 and later.

If NVDA add-ons are disabled, is your problem still occurring?

No add-ons installed.

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Did not attempt.

[seanbudd]

Tagging @LeonarddeR and @jcsteh who may have ideas on the cause

[jcsteh]

Some WASAPI function is returning an HRESULT of ERROR_TOO_MANY_OPEN_FILES (4, "The system cannot open the file."). That's a bit odd. What's also odd is that we only see this error code once. I would expect to see some error every time the synth tries to speak.

1. We check for the documented return codes of AUDCLNT_E_DEVICE_INVALIDATED and AUDCLNT_E_NOT_INITIALIZED from IAudioClient::GetCurrentPadding. Maybe we need to add ERROR_TOO_MANY_OPEN_FILES to that.
2. Or maybe we do get the right error code initially and then we get ERROR_TOO_MANY_OPEN_FILES when we try to reopen.
3. Does anyone know what happens to the default audio device when a remote desktop session disconnects? Understanding that would be helpful here.

[jcsteh]

What's also odd is that we only see this error code once. I would expect to see some error every time the synth tries to speak.

Oh. I guess the queue gets stuck if feed() ever throws an exception because the oneCore driver doesn't handle that. So maybe WasapiWavePlayer.feed should catch all exceptions and just log them.

[jcsteh]

@WestonThayer, would you mind testing with eSpeak? I'm not suggesting this as a solution, but it would just help me get some more data to diagnose here, since I think the eSpeak driver is a bit more resilient.

[WestonThayer]

Of course, no problem. I will give it a try tomorrow and report back.

[WestonThayer]

It does not reproduce with eSpeak. Nor with Microsoft Speech API Version 5 (SAPI). So only Windows OneCore Voices reproduce.

3. Does anyone know what happens to the default audio device when a remote desktop session disconnects? Understanding that would be helpful here.

I'm fairly sure there isn't an audio device by default on a VM. When in a remote desktop session, Windows create a Remote Audio device and sets it as the default. I believe it's removed when the remote session ends. I haven't tested in years, but there used to be an issue where NVDA (and other screen readers IIRC) had speech muted if started programmatically while no remote session was active, even after a remote session did begin. I could work around this by installing https://vb-audio.com/Cable/.

[jcsteh]

Okay. That's good. So I think the simple solution here is to make WasapiWavePlayer.feed catch exceptions and just log them. The C++ code already handles reopening as needed and the fact that eSpeak works would suggest that this is happening correctly. That is also consistent with the WinMM code: it swallows and logs exceptions. The only difference is that in that case, the Python code handled reconnecting on exception, but with WASAPI, that's handled in C++.

[seanbudd]

@jcsteh - do you plan on testing out the fix/approach?

[jcsteh]

I'm not in a position to test this myself, but I'll try to create a try build for @WestonThayer to test if that's feasible.

[jcsteh]

I wonder whether this is actually a bug in the Remote Desktop audio driver. ERROR_TOO_MANY_OPEN_FILES is error code 4. AUDCLNT_E_DEVICE_INVALIDATED is the error code i would expect to see, which is 0x88890004, AKA AUDCLNT_ERR(0x004). Maybe the driver is throwing the error code without the severity and facility here. In any case, this doesn't change anything for us, more just a curiosity.