Forward audio

[benwa]

Directing audio to the client would be pretty neat.

Currently you can accomplish the same thing by plugging a 3.5mm cable into the mic or line-in on the computer. Some manufacturers are dropping headphone ports though.

---

Suggestion by @rom1v: The best solution currently is to connect the device via bluetooth to the computer. It works for all apps, it does not suffer from additional limitations and works for basically all devices.

--> There is work in progress for Android >= 11. Please test #3757 🚀

[MrLint]

Adding this would really making streaming games would so much better. So far all the other existing solutions kinda suck.

[AdoPi]

Is it even possible without a custom rom?

[rom1v]

As suggested by Juan, one possibility to investigate is using AOA.

I already implemented something using HID over AOA in the past (I even have a publicly available sample), it should be quite similar.

[rom1v]

I just implemented a PoC (separated of scrcpy) to enable audio forwarding from the device to the computer: aoa-audio.

On Debian stable, it "works": the device sound is played on the computer.

This is quite manual though, and several issues must be solved before something similar could be implemented in scrcpy.

Any help is welcome 😉

[AdoPi]

May it be possible to use AudioRecord api server side?
Problem is that AudioRecord requires an Activity context, I don't know if we can spawn one by using app_process.

[rom1v]

AudioRecord may only record input sources (like microphone), not the device audio output.

[AdoPi]

I may be wrong but here is an example of what I thought:

https://github.com/tzutalin/ReadPCMData/blob/master/src/com/da/readpcmtest/MainActivity.java

Sorry I should have added this on my previous comment.

[rom1v]

Oh, you're right, I missed the REMOTE_SUBMIX audio source.

[rom1v]

Arf, just calling new AudioRecord(…) segfaults, probably because there is no Context (fault addr 0x0).

Note that AudioRecord itself does not require a Context explicitly.

[AdoPi]

Yes... :(
That's why I was wondering if we could create one even if we are using app process.

Audio support is really tricky :/

[rom1v]

Even from an app, the source REMOTE_SUBMIX does not work:

AudioRecord: Could not get audio input for record source 8, sample rate 44100, format 0x1, channel mask 0xc, session 73, flags 0

(while it works with MIC for example).

[rom1v]

Hi,

I would like to share with you my work-in-progress branch: audio. This branch can be rebased/rewritten at any time.

I implemented it, and it works fine for me! 🎉

I would like early feedbacks so that any major issue can be fixed.

Requirements

• Linux (for now, I have problems to make it work on Windows and MacOS)
• a device supporting AOA2
• the device must be connected via USB (not adb connect)

How to

Install libusb:

sudo apt install libusb-1.0-0 libusb-1.0-0-dev

Checkout the audio branch:

git fetch
git checkout origin/audio

Use a debug build to report problems:

meson d
ninja -C d

Run with audio forwarding enabled:

./run d --forward-audio

The audio should be forwarded.

When closed, the audio forwarding may not be totally disabled. It may require to restart the current audio track (in VLC for example). If it is not sufficient, unplug the device. At worst (but it should not be necessary), disable/enable USB debugging.

How it works

Scrcpy connects to the device over USB to send AOA requests so that the computer becomes an "USB audio accessory" for the device.

This creates a new audio input source on the computer (which can be seen in PulseAudio for example).

Scrcpy records this input source and immediately plays it to the default output source.

Thank you for your feedbacks.