Add option to reduce audio latency using scrcpy

[DerpyChap]

Waydroid appears to have a significant amount of audio latency, to the point where rhythm games become impossible to play unless you adjust the audio latency in the game's settings.

By using scrcpy, you can significantly reduce the amount of audio latency.

Using the game Cytoid as an example, here's the amount of measured end-to-end latency in the game's calibration tool when using Waydroid's audio playback:

When running scrcpy --audio-buffer 30 --audio-codec=raw --no-video-playback --tcpip=[WAYDROID_IP_ADDRESS]:5555 via SSH I measured the following latency:

For a rough idea of how this compares to an actual Android device, I ran the same test on my Sony Xperia 1 II.

This results in roughly half the end-to-end audio latency of Waydroid's native audio output, and brings things much closer to an actual Android device. I tried lower values for --audio-buffer, but it resulted in some noticeable popping in the audio. I haven't noticed any issues in more demanding games that I have tested, but despite that it might be best to make such a feature default off just in case.

[ryanrudolfoba]

i'd have to read up on it and wrap my head how to implement it on the script.

[ActualMandM]

Does 20 audio buffer crackle on your end? I tried it with Project Sekai and it worked fine for me. (plus it just manages to work with the game's -20/20 limit)

If it doesn't crackle, then it might be a good default for the buffer if this gets implemented.

[DerpyChap]

At 20 I noticed some infrequent popping, although I didn't notice any at 25.

[ryanrudolfoba]

I think i know how to add this to the script. Will there be any negative effect if default to 25?

[DerpyChap]

I didn't notice any issues when set to 25, but I haven't done any extensive testing.

[DerpyChap]

So I originally glanced over this in the original Waydroid issue since they said they had "little success", but in my testing it at gets audio latency down much closer to scrcpy.

Overlaying /system/etc/init/audio.rc containing the following:

 on init
    export PULSE_LATENCY_MSEC 60
    export PIPEWIRE_LATENCY 24/48000

reduces latency significantly. I tried setting PULSE_LATENCY_MSEC lower but it just resulted in there being no audio, although as far as I am aware the Steam Deck uses Pipewire for audio playback, so the PIPEWIRE_LATENCY environment variable should be the one that matters here? Admittedly I'm not all that familiar with how audio works on Linux so I'm not 100% certain there.

@ActualMandM how would you say this compares to scrcpy? Create a file at /var/lib/waydroid/overlay/system/etc/init/audio.rc with the contents above.

[ActualMandM]

The delay in the calibration is pretty bad with this still, even with it set to -20. I'm able to get PULSE_LATENCY_MSEC set to 50 on my end without audio being missing.

[ryanrudolfoba]

Here's my findings using scrcpy and cytoid -

without scrcpy - 0.429s offset
with scrcpy 30 buffer - 0.277s
with scrcpy 25 buffer - 0.293s
with scrcpy 20 buffer - 0.302s

So it looks like the sweet spot is the 30 buffer?
btw i noticed if scrcpy is enabled the audio becomes very faint compared to when its turned off, do you guys get the same behavior?

It could also be my version of scrcpy. The one from pacman repo is old 2.0.x and doesnt support the no-video-playback parameter. I build from git source to get the latest 2.3.x.

What version do you guys use for scrcpy and where did you get it?

[ryanrudolfoba]

here's how to have scrcpy launch automatically in Game Mode -

create a file - /usr/bin/waydroid-fix-audio and make the file executable. Contents of the file -
#!/bin/bash
scrcpy --audio-buffer 30 --audio-codec=raw --no-video-playback

Edit the file ~/Android_Waydroid/cage_helper.sh and add at the end -

sleep 5
/usr/bin/waydroid-fix-audio

Make sure USB debugging is enabled and always accept the connection from Steam Deck.

Please test and i can add this to the next version.

[ryanrudolfoba]

Using the overlay audio.rc method the cytoid offset shows 0.171s

I used this -
on init
export PULSE_LATENCY_MSEC 60
export PIPEWIRE_LATENCY 24/48000

So i guess this is the way to go? Much cleaner too as no extra binaries needed, and my audio is not faint compared to when scrcpy is enabled.

[DerpyChap]

So it looks like the sweet spot is the 30 buffer? btw i noticed if scrcpy is enabled the audio becomes very faint compared to when its turned off, do you guys get the same behavior?

It could also be my version of scrcpy. The one from pacman repo is old 2.0.x and doesnt support the no-video-playback parameter. I build from git source to get the latest 2.3.x.

What version do you guys use for scrcpy and where did you get it?

I installed it from nix, which I believe has the latest release. The audio sounded the same for me when using scrcpy compared to without, didn't notice anything sounding faint. I'd recommend installing the latest scrcpy release and re-testing.

[ActualMandM]

So it looks like the sweet spot is the 30 buffer? btw i noticed if scrcpy is enabled the audio becomes very faint compared to when its turned off, do you guys get the same behavior?

I had to turn the volume back up in settings once connecting scrcpy. I grabbed v2.1 from the Arch Linux archive because the latest version gave me an outdated glibc error.

[ActualMandM]

The delay in the calibration is pretty bad with this still, even with it set to -20. I'm able to get PULSE_LATENCY_MSEC set to 50 on my end without audio being missing.

I probably should've test it with it set to 0, whoops! It seems to sync up pretty well once doing that. Will need to test with a song though to be sure.

EDIT: Testing with a song, I still need the delay set to -20 but at the very least I don't need scrcpy to get that result now (both with PULSE_LATENCY_MSEC set to 60 and 50). I think the audio.rc method might be the way to go.

[ryanrudolfoba]

I'm leaning on the audio.rc method too as no extra binaries needed.

[DerpyChap]

Yeah, using the audio.rc file seems to be the most ideal solution. Don't think there's anything else that can reduce latency further without reworking how Waydroid handles audio altogether.

[ryanrudolfoba]

Awesome. What willl be the best default values to include on the script?

Moving forward I plan to include a "waydroid toolbox" that user runs in Desktop Mode if they want to change the default values. It will just be a simple zenity-based menu and user can choose -

disable / enable custom host file
toggle between libndk / libhoudini
delete current android image and reinitialize a new one
disable / enable audio latency fix

[DerpyChap]

I'd probably go with either 32/48000 or 64/48000 since that gives us a bit of a buffer (around 1.2ms with 64) which could help reduce the likelihood of audio popping from high CPU loads.

PULSE_LATENCY_MSEC seems to have a hard limit of 50-60 (haven't tested 50 with it, myself), although I'm not sure if that option is doing much on the Deck anyway.

[ryanrudolfoba]

50msec and 32/48000 resulted in 0.164s. sweet!

[ryanrudolfoba]

pushed the proposed config and updated the installer script.

[ryanrudolfoba]

Awesome. What willl be the best default values to include on the script?

Moving forward I plan to include a "waydroid toolbox" that user runs in Desktop Mode if they want to change the default values. It will just be a simple zenity-based menu and user can choose -

disable / enable custom host file toggle between libndk / libhoudini delete current android image and reinitialize a new one disable / enable audio latency fix

preliminary Waydroid Toolbox is now here on the latest version

at the moment cant implement toggle between libhoudini / libndk - the var partition is tiny libhoudini doesnt fit. libndk is small so we are good with that.