FR - Add pipewire video source(s) in place of v4l2

[pallaswept]

• I have checked that a similar feature request does not already exist.

Is your feature request related to a problem? Please describe.
scrcpy currently can use v4l2loopback devices as virtual cameras to display the screen and share it to other applications. It works as expected.

This is somewhat limited as it means the device must have a fixed resolution, so it is not possible to rotate the display, for example.

Additionally, and worse, v42loopback clients (all webcam apps, chrome browser, firefox* etc) have an issue here, where applications which expect a webcam as input often filter by devices which only have output capabilities, and not capture capabilities. Accordingly, v4l2loopback has an option exclusive_caps,which makes the virtual device's capabilities advertise exclusively as a capture device, until it is fed input (eg by scrcpy) and then the device changes its capabilities to an output device, so that it will be recognised as a webcam by the client applications.

There is presently a bug in pipewire which means that feature cannot be used. I have reported that bug to the pipewire developers, however they are not in a rush to fix it, as they consider it a fix for an obsolete method - their opinion is that v4l2loopback devices should no longer be necessary at all, as applications (such as scrcpy) can provide a native pipewire source, which can then be consumed by clients. To quote the pipewire dev on this matter:

the ideal scenario is that [the application, scrcpy in this case] creates a pipewire source node, the browser creates a pipewire capture stream and wireplumber just links them together... no V4L2 involved.

Other developers appear to agree - Firefox already has support for consumption of webcam video from pipewire sources (media.webrtc.camera.allow-pipewire), chrome has it in development versions and it will be released soon (https://bugs.chromium.org/p/webrtc/issues/detail?id=13177), webcamoid supports pipewire sources, cheese will use pipewire in future.... and for legacy applications, there is pw-v4l2 provided by pipewire, which works with most webcam apps already, here's a gnome developer's blog where he uses the tool to wrap the cheese webcam app

So, while it would be good if pipewire is fixed up so that it doesn't have problems with v4l2loopback, the future of these things appears to be that applications providing video streams, such as scrcpy, do so by means of a native pipewire source - and it comes with the additional bonus of allowing us to rotate the android screen, so that would be nice. Perhaps we would even be able to provide both screen and camera sources?

Describe the solution you'd like
Provide a native pipewire source to output video from scrcpy, in place of v4l2loopback.

Describe alternatives you've considered
No alternatives that I am aware of.

Additional context
I think I covered it all, but please let me know if there's any further info you require.

[rom1v]

Yes, that would be great. 👍

[9thyear2]

found a workaround (for obs at least)

1. start up the video / audio capture however you like (but make sure playback is enabled), in my case i use this

scrcpy --video-source=camera --camera-id=0 --camera-size=1920x1080 --video-bit-rate=20M --v4l2-sink=/dev/video0 --video-codec=h264 --video-encoder=OMX.qcom.video.encoder.avc --max-fps=30 --no-audio

2. then create a new source using pipewire screen capture, and select the playback as the video source

(if you want the audio from the playback as well, then enable the audio and add it to obs using jack or pulse (whichever floats your boat), to isolate it from the desktop audio)

the downside is you cant minimize the window (doing so freezes the capture), however it seems i can just put it on a separate workspace and act like that workspace doesn't exist

makes me wonder if the devs can somehow abuse the playback mechanism they have along with pipewire video, to make a pipewire output where the window is hidden, and have pipewire capture the hidden playback

[titaniumtraveler]

makes me wonder if the devs can somehow abuse the playback mechanism they have along with pipewire video, to make a pipewire output where the window is hidden, and have pipewire capture the hidden playback

A wayland compositor would be a better place to do that.
At least theoretically making a small compositor that feeds all windows directly into pipewire instead of displaying them should be not too hard.

[piotr-dobrogost]

Btw, I'm observing that sometimes the v4l2loopback device with exclusive_caps=1 used with scrcpy gets stuck with Video Output capability and subsequently starting scrcpy does not change it to Video Capture anymore. I raised this in my comment at v4l2loopback/v4l2loopback#442 (comment).

Also, I believe here

Accordingly, v4l2loopback has an option exclusive_caps,which makes the virtual device's capabilities advertise exclusively as a capture device, until it is fed input (eg by scrcpy) and then the device changes its capabilities to an output device, so that it will be recognised as a webcam by the client applications.

@pallaswept has it backwards. Names of capabilities in v4l2 reflect the application side, not a device side. Thus Video Output means an application can output to such a device and Video Capture means an application can capture video from such a device.

This can easily be verified by running watch -n1 v4l2-ctl -d /dev/video2 --info and then starting scrcpy with --v4l2-sink=/dev/video2 and observing how advertised capability changes from Video Output to Video Capture.

[pallaswept]

@pallaswept has it backwards. Names of capabilities in v4l2 reflect the application side, not a device side.

This stems from being copypasted from the pipewire issue, where I was explaining it in common terms for a general audience. I can't assume that the person reading it will know v4l2-specific terminology, but if they do, they will already understand regardless of semantics. My patch works, so I'm pretty confident I don't have it backwards, although clearly, I'm not good at explaining it :) It's a no-win, though. If I use the technically correct terms, it will trick everyone who doesn't have strong familiarity with the inner workings of v4l2 (read: practically everyone). Since the punchline is the same (make a pipewire source) and it's now explained down here, I won't bother to edit it. I think everyone understands what's going on here :)