Sound skips and rattles heavily from Windows to Linux (xrdp) or Windows Server (RDS)

[vscfreire]

Using the latest FreeRDP 2 builds (build 648) and connecting from Windows to either Linux (xrdp 0.9.10) or Windows Server (RDS 2016) results in sound skipping and rattling frequently. This problem is present for some time now.

I’m using the following command for the connection with the Windows and Linux server:
wfreerdp.exe /v:hostname /cert-tofu /u:Administrator /p:password /sound:format:0x0001 /w:1000 /h:600

Several optimization tactics, like sound formats, quality, latency and network detection were used with very limited success, usually only delaying the sound skipping and rattling.

Also, using sound formats (e.g. WAVE_FORMAT_MPEGLAYER3 0x0055) other than PCM (0x0001) while connecting to Linux (xrdp 0.9.10) server will result in client immediately exiting when playing any kind of sound with [com.freerdp.client.common] - Network disconnect. Debugging shows “rdpsnd_virtual_channel_client_thread reported an error. Error was 13”

[akallabeth]

@vscfreire

• for some time now is how long? last commit there was ages back. (we´d appreciate some windows maintainer/tester for early feedback)
• Could you describe your setup with a bit more detail? I sometimes use the windows client and did not have that particular problem (but most of the time just listen to some video in background)
• Your 'optimization is not really one, better not do that (format` option that is). (most of the time that only leads to more issues, should really remove that option ;))
• the xrdp mp3 abort is expected, you can´t use a codec that is not compiled into the client. (and it is part of the EXPERIMENTAL ones, so they are buggy anyway) (not really a bug as the only way you can force this is by forceful command line override)

[akallabeth]

@vscfreire just tried the current nightly build https://ci.freerdp.com/job/freerdp-nightly-windows/arch=win64,label=vs2013/lastSuccessfulBuild/artifact/build/Release/wfreerdp.exe against various servers (rds and direct connections) and did not experience rattling or similar.
Could you provide a short description of your test routine to allow reproducing this?

[akallabeth]

@vscfreire Ok, did finally manage to reproduce, it was the missing latency estimation.
This still has some small glitches when reconnecting to a e.g. browser running a youtube video but the problems occurring with 'rattles' seem to be fixed.

[vscfreire]

@akallabeth Connecting to either Windows or Linux server seems much better now (with PCM). In both cases, sometimes (~20%) the sound starts perfect but after a minute or more the skips and rattles come back. If I stop and start the clip/video again it appears to fix it.

There's another situation, if I omit the sound format parameter to force PCM, the connection to Windows Servers 2016 (from a Windows 10) the FreeRDP client states: Opening device with format WAVE_FORMAT_AAC_MS [backend WAVE_FORMAT_PCM]. In this case I get sound distortion/rattle from the beginning (but no skipping).

In terms of environment, fairly standard Windows 10 (1903) on the client side, and a low latency, high throughput WAN connection to the server side ( Linux xrdp 0.9.10 or Windows Server RDS 2016). The WAN connection it's over IPSEC with 0% packet loss and 6 ms average rtt.

[akallabeth]

@vscfreire

• hmm, how do you get AAC support compiled in? our windows builds do not link against any AAC decoder, in that case better check your buildconfig.
• Could you check if the issues stop if you always wait until a packet is played (remove the while(winmm->diff == 0... check in channels/rdpsnd/client/winmm/rdpsnd_winmm.c)?

[vscfreire]

@akallabeth

• I have compiled freerdp with ffmpeg in windows to test different codecs. That's also how I was testing MP3 with Linux XRDP server. I can test other codecs in the Windows client if you like.

• I tried removing the cycle and it gets more prone to rattling. However I noticed that without it, the lip sync is better.

[akallabeth]

@vscfreire ok. some notes:

• Never really got mp3 running with xrdp and have no windows 7 that announces it (therefore it is experimental)
• Which AAC decoder backend are you using? Have had success with ffmpeg and faad under linux, but windows never tried.

[vscfreire]

@akallabeth
Was trying to find a way to help you reproduce the problems and introducing even a very small packet loss (1% or 0,5%) in the connection triggers it. There's a really simple utility to simulate several network conditions on Windows: https://jagt.github.io/clumsy/.

With Windows, I'm using the ffmpeg backend, but just tried using faad in Linux and the sound works great connecting to Windows Server, even with induced latency and drops. Also tried PCM in Linux and it's also really resilient to network conditions.

[akallabeth]

@vscfreire ok, so it is the winmm backend that is (still) not working well :/ (the rest of the path is the same on all os)

[akallabeth]

@vscfreire Ok, is the current nightly working? Have run it a few times and the problems seem fixed.

[akallabeth]

Can not reproduce with nightly build https://ci.freerdp.com/job/freerdp-nightly-windows/656/

[vscfreire]

Sorry for the late reply.

I'm still getting problems, when using the latest nightly build (656) on Windows (x86_64).

Tried the latest nightly on Linux (connecting from the same network to the same Windows Server 2016 machine) and the sound is just perfect, spot-on.

Is there any additional info I can provide to help debug the issue?

[vscfreire]

Just to complement what I said regarding the latest nightly (running on Windows)...

Sometimes I connect and the sound is perfectly synced and with no skipping/rattling. But if I disconnect and reconnect to the same session, the sound is now skipping/rattling again. And it stays that way until I disconnect. Sometimes this reconnect makes things worse, sometimes it improves a bit (and sometimes it becomes perfect).

[vscfreire]

In the latest build (Nightly build 710, win32, vs2013) the sound is always skipping. The last time I tested, some times it worked but in the latest build, I can't get proper sound anytime. (windows 10 desktop to windows 2016 server)

[akallabeth]

Ok, #5818 should (finally) fix this one. please confirm

[vscfreire]

@akallabeth Great work! No skips, no rattles, no crashes. This is based only on an hour test but I think you can close this issue. Thanks for your support.

[akallabeth]

@vscfreire good news, finally :)