Can't seem to get Mac OS X Audio working with FreeRDP

[Nexarian]

Describe the bug

MACAUDIO backend seems to be broken on Mac OS X (Catalina 10.15.7). I can clearly see the audio buffer being received in the logs, but there is no sound period. This happens with both an XRDP backend (Where the Mac OS Remote Desktop client has sound) as well as when I connect to an up-to-date Windows 10 installation (Where also, the Microsoft client has working sound).

I've tried changing the device (/sound:dev:1-6), as well as playing with multiple sys options. No sound. Sometimes, and very rarely, I'll hear the sound but it's very scattered and jittery and not good.

To Reproduce
Build with the following:

git clone https://github.com/FreeRDP/FreeRDP.git

cd FreeRDP

export PKG_CONFIG_PATH=$(brew --prefix)/opt/openssl/lib/pkgconfig

cmake -DWITH_X11=ON DWITH_JPEG=ON -DBUILD_SHARED_LIBS=ON -DWITH_OPENH264=ON -DWITH_CAIRO=ON -DCHANNEL_RDPSND=ON -DCHANNEL_RDPSND_CLIENT=ON -DWITH_SPEEX=OFF -DWITH_MACAUDIO=ON -DWITH_PULSE=OFF -DWITH_FFMPEG=ON -DWITH_DSP_FFMPEG=ON -DWITH_ALSA=OFF -DCMAKE_OSX_ARCHITECTURES:STRING=x86_64 .

cmake --build .

sudo make install

/usr/local/bin/xfreerdp /bpp:32 /v:[hostname]:3389 /network:lan /sound /u:[username] /p:[password] /gfx +glyph-cache +offscreen-cache +bitmap-cache /video +clipboard +compression +fonts /dynamic-resolution /w:2560 /h:1396

When I add /log-level:debug, it's clear there was a lot of buffering as I see a lot of sound messages, like the following:

[15:00:53:879] [2530:02d03000] [DEBUG][com.freerdp.channels.rdpsnd.client] - [static] WaveInfo: cBlockNo: 108 wFormatNo: 2 [WAVE_FORMAT_AAC_MS]
[15:00:53:879] [2530:02d03000] [DEBUG][com.freerdp.channels.rdpsnd.client] - [static] Wave: cBlockNo: 108 wTimeStamp: 62644, size: 279

But no sound plays

This is FreeRDP version 3.0.0-dev (5fc2290c0)
Build configuration: BUILD_TESTING=OFF BUILTIN_CHANNELS=ON HAVE_AIO_H=1 HAVE_EXECINFO_H=1 HAVE_FCNTL_H=1 HAVE_FLAG_SEARCH_PATHS_FIRST=1 HAVE_GETLOGIN_R=1 HAVE_INTTYPES_H=1 HAVE_MATH_C99_LONG_DOUBLE=1 HAVE_POLL_H=1 HAVE_PTHREAD_MUTEX_TIMEDLOCK_LIB= HAVE_PTHREAD_MUTEX_TIMEDLOCK_LIBS= HAVE_PTHREAD_MUTEX_TIMEDLOCK_SYMBOL= HAVE_STRNDUP=1 HAVE_SYSLOG_H=1 HAVE_SYS_EVENTFD_H= HAVE_SYS_FILIO_H=1 HAVE_SYS_MODEM_H= HAVE_SYS_SELECT_H=1 HAVE_SYS_SOCKIO_H=1 HAVE_SYS_STRTIO_H= HAVE_SYS_TIMERFD_H= HAVE_TM_GMTOFF=1 HAVE_UNISTD_H=1 HAVE_XI_TOUCH_CLASS= WITH_ALSA=OFF WITH_CAIRO=ON WITH_CCACHE=ON WITH_CHANNELS=ON WITH_CLANG_FORMAT=ON WITH_CLIENT=ON WITH_CLIENT_AVAILABLE=1 WITH_CLIENT_CHANNELS=ON WITH_CLIENT_CHANNELS_AVAILABLE=1 WITH_CLIENT_COMMON=ON WITH_CLIENT_INTERFACE=OFF WITH_CUPS=ON WITH_DEBUG_ALL=OFF WITH_DEBUG_CAPABILITIES=OFF WITH_DEBUG_CERTIFICATE=OFF WITH_DEBUG_CHANNELS=OFF WITH_DEBUG_CLIPRDR=OFF WITH_DEBUG_DVC=OFF WITH_DEBUG_KBD=OFF WITH_DEBUG_LICENSE=OFF WITH_DEBUG_MUTEX=OFF WITH_DEBUG_NEGO=OFF WITH_DEBUG_NLA=OFF WITH_DEBUG_NTLM=OFF WITH_DEBUG_RAIL=OFF WITH_DEBUG_RDP=OFF WITH_DEBUG_RDPDR=OFF WITH_DEBUG_RDPEI=OFF WITH_DEBUG_RDPGFX=OFF WITH_DEBUG_REDIR=OFF WITH_DEBUG_RFX=OFF WITH_DEBUG_RINGBUFFER=OFF WITH_DEBUG_SCARD=OFF WITH_DEBUG_SND=OFF WITH_DEBUG_SVC=OFF WITH_DEBUG_SYMBOLS=OFF WITH_DEBUG_THREADS=OFF WITH_DEBUG_TIMEZONE=OFF WITH_DEBUG_TRANSPORT=OFF WITH_DEBUG_TSG=OFF WITH_DEBUG_TSMF=OFF WITH_DEBUG_TSMF=OFF WITH_DEBUG_TSMF_AVAILABLE=0 WITH_DEBUG_URBDRC=OFF WITH_DEBUG_WND=OFF WITH_DEBUG_X11=OFF WITH_DEBUG_X11_CLIPRDR=OFF WITH_DEBUG_X11_LOCAL_MOVESIZE=OFF WITH_DEBUG_XV=OFF WITH_DSP_EXPERIMENTAL=OFF WITH_DSP_FFMPEG=ON WITH_FAAC=OFF WITH_FAAD2=OFF WITH_FFMPEG=TRUE WITH_FFMPEG=TRUE WITH_FUSE=OFF WITH_GFX_H264=ON WITH_GPROF=OFF WITH_GSM=OFF WITH_GSSAPI=OFF WITH_ICU=OFF WITH_JPEG=OFF WITH_LAME=OFF WITH_LIBRARY_VERSIONING=ON WITH_LIBSYSTEMD=OFF WITH_MACAUDIO=ON WITH_MACAUDIO_AVAILABLE=1 WITH_MANPAGES=ON WITH_MBEDTLS=OFF WITH_OPENCL=OFF WITH_OPENH264=TRUE WITH_OPENH264=TRUE WITH_OPENH264_LOADING=OFF WITH_OPENSSL=ON WITH_PCSC=ON WITH_PROFILER=OFF WITH_PULSE=OFF WITH_SAMPLE=OFF WITH_SANITIZE_ADDRESS=OFF WITH_SANITIZE_ADDRESS_AVAILABLE=1 WITH_SANITIZE_MEMORY=OFF WITH_SANITIZE_MEMORY_AVAILABLE=1 WITH_SANITIZE_THREAD=OFF WITH_SANITIZE_THREAD_AVAILABLE=1 WITH_SERVER=OFF WITH_SERVER_INTERFACE=ON WITH_SMARTCARD_INSPECT=OFF WITH_SOXR=OFF WITH_SPEEX=OFF WITH_SSE2=ON WITH_SWSCALE=OFF WITH_THIRD_PARTY=OFF WITH_VAAPI=OFF WITH_VALGRIND_MEMCHECK=OFF WITH_VALGRIND_MEMCHECK_AVAILABLE=1 WITH_WINPR_TOOLS=ON WITH_X11=ON WITH_X264=OFF WITH_XCURSOR=ON WITH_XEXT=ON WITH_XFIXES=ON WITH_XI=OFF WITH_XINERAMA=ON WITH_XKBFILE=ON WITH_XRANDR=ON WITH_XRENDER=ON WITH_XSHM=ON WITH_XV=ON WITH_ZLIB=ON
Build type:          Release
CFLAGS:               -Wno-unused-parameter -Wno-unused-macros -Wno-padded -Wno-c11-extensions -Wno-gnu -Wno-unused-command-line-argument -Wno-deprecated-declarations -fno-omit-frame-pointer -DWINPR_DLL
Compiler:            Clang, 12.0.0.12000032
Target architecture: x64

[Nexarian]

Note that my system has a lot of sound devices, including every single monitor connected to it (over DisplayPort) -- Does FreeRDP make sure it's correctly reporting to the right sound device? Is there a way to double check that?

[akallabeth]

@Nexarian I´d start with a test without FFMPEG so that you get plain PCM data.
Just ran that on my mac test machine and the sound is ok for both, PCM and FFMPEG AAC (against a windows 10 test machine)
So I can conclude the sound backend is not at issue here, but rather how the data is processed.

[Nexarian]

I also have it working on my Mac now, but there are still problems. I have my macbook attached to a CalDigit TS3+ docking station, and the audio only outputs through that. If I switch to the native MacBook speakers, it goes silent. I suspect it's defaulting to whatever device "0" is, and ignoring the default Mac OS X selection. This also means that bluetooth headphones are also silent.

I think the problem is definitely device redirection. I'm not sure what FreeRDP intends to support here, but one hacky solution might be to have a CLI option that causes FreeRDP to output the list of devices and what they correspond to, and then give you a command line flag that's appropriate for redirection to the proper device.

The other weird thing, is I have to select the CalDigit device as the default output on Mac OS to hear anything from FreeRDP. You'd think that sound would play through the CalDigit regardless of what is selected if FreeRDP is hard-outputting to it.

For those who may be wondering how exactly to get it working though, these are the build flags I used. If cmake throws an error for missing headers, I used Homebrew to install the proper package.

Note: Don't use Homebrew itself to install FreeRDP as it doesn't have the WITH_MACAUDIO option enabled.

cmake -DWITH_X11=ON DWITH_JPEG=ON -DBUILD_SHARED_LIBS=ON -DWITH_OPENH264=ON -DWITH_CAIRO=ON -DCHANNEL_RDPSND=ON -DCHANNEL_RDPSND_CLIENT=ON -DWITH_SPEEX=ON -DWITH_MACAUDIO=ON -DWITH_PULSE=OFF -DWITH_FFMPEG=ON -DWITH_DSP_FFMPEG=ON -DWITH_ALSA=OFF -DCMAKE_OSX_ARCHITECTURES:STRING=x86_64 .

[Nexarian]

After digging into the code more, the code here https://github.com/FreeRDP/FreeRDP/blob/master/channels/rdpsnd/client/mac/rdpsnd_mac.m#L147

Specifies not the default output the device, but instead the "Play sound effects through" device, see this Stackoverflow link:

https://stackoverflow.com/questions/58908794/what-is-the-difference-between-defaultsystemoutputdevice-and-defautoutputdevice

Indeed, I tested this. When I change to kAudioHardwarePropertyDefaultOutputDevice and rebuild, now the default system output device is used.

I'm guessing the Play sound effects through device was not the intended use.

[Nexarian]

After further testing, this works, but it's not complete. This doesn't adapt to changes in the Mac OS X default system output device. FreeRDP should probably default to that behavior unless a device ID is explicitly set for sound, which should probably also be supported. I don't fully understand the way the AVAudioEngine for Mac OS X is supposed to work here, and examples on the web seem strangely scarce. Here are some breadcrumbs:

https://stackoverflow.com/questions/26070058/how-to-get-notification-if-system-preferences-default-sound-changed
https://stackoverflow.com/questions/9674666/default-audio-output-getting-device-changed-notification-coreaudio-mac-os-x
https://gist.github.com/robotconscience/490ee50d8ad75b47ea1f

[greenvilleassociates]

Naxarian... quick question.. I followed your build instructions and they worked for me too.... but I am getting a display error not defined now... also how do you verify freerdp is working and on what port. Lastly can you change the port to something different?