Repository navigation
Forward audio #14
Description
Activity
Adding this would really making streaming games would so much better. So far all the other existing solutions kinda suck.
Reacted by DeedleFake, Tim Greller, Julian Jarecki, AzureRaptor, RangaRaoPulipaka, Mathias Kivi, davepee, a-felix, Theodore, Deepansh Singh Arya and 17 moreIs it even possible without a custom rom?
I just implemented a PoC (separated of scrcpy) to enable audio forwarding from the device to the computer: aoa-audio.
On Debian stable, it "works": the device sound is played on the computer.
This is quite manual though, and several issues must be solved before something similar could be implemented in scrcpy.
Any help is welcome 😉
Reacted by Adonis, Jinhui.Lin, EVGENIY PAVLIKOV, Vedant Paranjape, illusiony, Vaibhav, nakhla, webian, Fazle Rabbi, AzureRaptor and 10 moreReacted by ℂ𝕠𝕠𝕠𝕝 (𝕘𝕚𝕥𝕙𝕦𝕓.𝕔𝕠𝕞/ℂ𝕠𝕠𝕠𝕝)May it be possible to use AudioRecord api server side?
Problem is that AudioRecord requires an Activity context, I don't know if we can spawn one by using app_process.AudioRecordmay only record input sources (like microphone), not the device audio output.Reacted by ℂ𝕠𝕠𝕠𝕝 (𝕘𝕚𝕥𝕙𝕦𝕓.𝕔𝕠𝕞/ℂ𝕠𝕠𝕠𝕝) and terrawolf34I may be wrong but here is an example of what I thought:
https://github.com/tzutalin/ReadPCMData/blob/master/src/com/da/readpcmtest/MainActivity.java
Sorry I should have added this on my previous comment.
Reacted by ℂ𝕠𝕠𝕠𝕝 (𝕘𝕚𝕥𝕙𝕦𝕓.𝕔𝕠𝕞/ℂ𝕠𝕠𝕠𝕝)Oh, you're right, I missed the
REMOTE_SUBMIXaudio source.Reacted by Adonis, illusiony, nakhla, xtmhm2000, minhducsun2002, AzureRaptor, NishuTyagi2, Lena and myfreaxReacted by ℂ𝕠𝕠𝕠𝕝 (𝕘𝕚𝕥𝕙𝕦𝕓.𝕔𝕠𝕞/ℂ𝕠𝕠𝕠𝕝) and terrawolf34Arf, just calling
new AudioRecord(…)segfaults, probably because there is noContext(fault addr 0x0).Note that
AudioRecorditself does not require aContextexplicitly.Yes... :(
That's why I was wondering if we could create one even if we are using app process.Audio support is really tricky :/
Even from an app, the source
REMOTE_SUBMIXdoes not work:AudioRecord: Could not get audio input for record source 8, sample rate 44100, format 0x1, channel mask 0xc, session 73, flags 0(while it works with
MICfor example).Reacted by ℂ𝕠𝕠𝕠𝕝 (𝕘𝕚𝕥𝕙𝕦𝕓.𝕔𝕠𝕞/ℂ𝕠𝕠𝕠𝕝)Reacted by Adonis, AzureRaptor, Lena and SnowHi,
I would like to share with you my work-in-progress branch:
audio. This branch can be rebased/rewritten at any time.I implemented it, and it works fine for me! 🎉
I would like early feedbacks so that any major issue can be fixed.
Requirements
- Linux (for now, I have problems to make it work on Windows and MacOS)
- a device supporting AOA2
- the device must be connected via USB (not
adb connect)
How to
Install libusb:
sudo apt install libusb-1.0-0 libusb-1.0-0-devCheckout the
audiobranch:git fetch git checkout origin/audioUse a debug build to report problems:
meson d ninja -C dRun with audio forwarding enabled:
./run d --forward-audioThe audio should be forwarded.
When closed, the audio forwarding may not be totally disabled. It may require to restart the current audio track (in VLC for example). If it is not sufficient, unplug the device. At worst (but it should not be necessary), disable/enable USB debugging.
How it works
Scrcpy connects to the device over USB to send AOA requests so that the computer becomes an "USB audio accessory" for the device.
This creates a new audio input source on the computer (which can be seen in PulseAudio for example).
Scrcpy records this input source and immediately plays it to the default output source.
Thank you for your feedbacks.
Reacted by Adonis, Amaël (r4dixx) Sikel, Albert Armea, Wish, Dee, numbermaniac, Nixon Nelson Paliakkara, Caio Costa 🇧🇷🇵🇹, Paola (Alma) Captanovska, Miguel Hernández and 35 more266 remaining items
I added binaries for Windows if you want to test: #3757
Tested binary on 64bit Windows. Worked flawlessly on Android 12 Samsung device! There's slight delay in audio and video (video lags slightly which kind of irks for sometime when you observe lip-sync) but good nevertheless. It probably is because I was on wifi adb.
Implemented in scrcpy v2.0 🚀
Reacted by Luni, Jared Skinner, Mohit, Louis CAD, Michaël Veillet Guillem, fabianski7, Fareez Iqmal, Bennett Blodinger, deskid, Richardson Rosa and 4 moreReacted by Luni and Jared SkinnerReacted by Luni, Jared Skinner, Bennett Blodinger, keze87, Richardson Rosa, dtaddis, xiaoke and qaz1qazlol2Reacted by Luni, Jared Skinner, Michaël Veillet Guillem, Nicolás, Adek Maulana and Unal KalkanReacted by Luni, Dovie Weinstock, rp1231, mmis1000, Louis CAD, Bennett Blodinger, dtaddis and K. N.Reacted by Too large to fit in the margin and Corbin CrutchleyHow should I make audio forwarding disabled by default?
Reacted by Too large to fit in the marginRun
scrcpy --no-audiofrom a script (for Windows, see the end of https://github.com/Genymobile/scrcpy/blob/master/doc/windows.md).Reacted by Too large to fit in the marginHow do I play audio on both the headset and the receiver? I want the beatsaber audio to be perfectly lined up when playing, but also to be able to stream it.
Not possible (for now?), see #3875.
It must be possible via some set of configurations on the host device; the oculus does this by default for all of it's streaming. Is there a way to "record" device audio and play it back on the second device, maybe if there's something additional installed on the Android side?
Sorry, but how does it work on scrcpy 2.0. I tried connecting via USB, the phone shows, visually speaking, but any sound from the phone, happens on the phone, not on the PC. The PC is Ubuntu 23.04
What is the console output when you run scrcpy? What is your Android version?
scrcpy 2.0 https://github.com/Genymobile/scrcpy
/usr/local/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 8.2 MB/s (52867 bytes in 0.006s)
[server] INFO: Device: samsung SM-S908U1 (Android 13)
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 530.41.03
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2312@rom1v Sorry forgot about that. Thank you.
What is the result of
scrcpy -Vdebug?any sound from the phone, happens on the phone, not on the PC
For example, if you play a video on your phone with vlc (or any other player), the sound is played on the phone?
Note that notifications or phone calls are not forwarded.
scrcpy 2.0 https://github.com/Genymobile/scrcpy
DEBUG: ADB device found:
DEBUG: --> (usb) R5CT408VMPT device SM_S908U1
DEBUG: Device serial: R5CT408VMPT
DEBUG: Using server: /usr/local/share/scrcpy/scrcpy-server
/usr/local/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 6.0 MB/s (52867 bytes in 0.008s)
[server] INFO: Device: samsung SM-S908U1 (Android 13)
DEBUG: Server connected
DEBUG: Starting controller thread
DEBUG: Starting receiver thread
[server] DEBUG: Using audio encoder: 'c2.android.opus.encoder'
INFO: Renderer: opengl
INFO: OpenGL version: 4.6.0 NVIDIA 530.41.03
INFO: Trilinear filtering enabled
DEBUG: Using icon: /usr/local/share/icons/hicolor/256x256/apps/scrcpy.png
DEBUG: Demuxer 'video': starting thread
DEBUG: Demuxer 'audio': starting thread
[server] DEBUG: Using encoder: 'c2.qti.avc.encoder'
INFO: Initial texture: 1080x2312
DEBUG: [Audio] Buffer underflow, inserting silence: 240 samples
DEBUG: [Audio] Buffer underflow, inserting silence: 240 samples
DEBUG: [Audio] Buffer underflow, inserting silence: 240 samples
DEBUG: [Audio] Buffer underflow, inserting silence: 240 samples
You helped me solve the mystery. I was trying to get phone calls on the PC.
For videos I can literally hear them on the PC which IS AMAZING!! This is like a new level for me. I am guessing on version 3.0 they might be able to include phone calls so I can do full productivity on the PC without the phone.
Reacted by K. N. and Rui PimentelReacted by K. N.
Directing audio to the client would be pretty neat.
Currently you can accomplish the same thing by plugging a 3.5mm cable into the mic or line-in on the computer. Some manufacturers are dropping headphone ports though.
Suggestion by @rom1v: The best solution currently is to connect the device via bluetooth to the computer. It works for all apps, it does not suffer from additional limitations and works for basically all devices.
--> There is work in progress for Android >= 11. Please test #3757 🚀