[rfc/wip] split device #359

vicamo · 2020-10-15T17:21:52Z

Here is my initial attempt to try implementing device split. Still in the very early stage, but I'd like to have your opinions before jumping too far or in the wrong direction.

The main idea/goal is to:

remove single device code completely.

Single device is the root of all evil so far, and there probably has no reason to do this, especially switching device capabilities at runtime. This can, probably has to, be done after the transition. Module parameter exclusive_caps will be removed; assign identical device number to output and capture devices simultaneously becomes illegal.
keep the ability to instantiate multiple devices through module parameters.

Currently module parameter output_nr is added for this purpose.
rewrite buffer management with video videobuf2.
embed video_device in v4l2_loopback_device

This can simplify many struct pointer resolutions and memory leaks.

umlaeute · 2020-10-19T11:59:03Z

keep the ability to instantiate multiple devices through module parameters. (output_nr)

ACK

rewrite buffer management with video videobuf2.

fully ACK ;-)

embed video_device in v4l2_loopback_device

I don't care

remove single device code completely.

ah well.
let's put it like this:

if removing single-device code completely simplifies the code a lot and makes implementation just a lot easier, go for it.
if otoh, allowing single-device could be accomplished with just a few more (output_dev != capture_dev) lines i would rather keep it.

Single device is the root of all evil so far, and there probably has no reason to do this,
especially switching device capabilities at runtime.

not sure i understand this

This can, probably has to, be done after the transition.

not sure i know what "this" and "the transition" refer to right now (if "this" is "getting rid single-device support" and the "transition" is "dynamic device management" then you are probably right.
(but then: i think "dynamic device management" is ready to be rolled out; so this transition is basically done)

Module parameter exclusive_caps will be removed;

in all cases: ACK
even if we still support single-device at the end of the day, the exclusive_caps should go away
(in the case of split-devices it is not needed; in the case of single-device it can always be set to false)

assign identical device number to output and capture devices simultaneously becomes illegal.

obviously this is (only) an implication of disallowing single-device.

so: go ahead ;-)

janakelarsson · 2021-01-13T07:42:15Z

Is this still WIP or has it stalled?

vicamo · 2021-01-15T06:02:25Z

Is this still WIP or has it stalled?

Hi, sorry, mostly stalled as we focus on the stable release 0.12.5 to implement camera relay.

janakelarsson · 2021-01-15T08:08:48Z

I would contribute but the effort to grok existing code is probably prohibitive.

I am looking forward to not having to restart Zoom each time I pull out the green screen.

umlaeute · 2021-01-15T08:39:12Z

I would contribute but the effort to grok existing code is probably prohibitive.

cool, go ahead ;-)

I am looking forward to not having to restart Zoom each time I pull out the green screen.

however, here zoom works without split-devices; so i wonder why you think you need it.

janakelarsson · 2021-01-15T08:42:03Z

I don't have OBS Studio Virtual Cam running all the time, it eats CPU.

Zoom does not detect a newly started Virtual Cam, so I need to restart Zoom. Teams too. After restart they do find the Virtual Cam.

janakelarsson · 2021-01-19T22:15:45Z

I looked at the code. Besides one obvious place (line 2307) where "device->" should be "dev->" I cannot help much, apologies.

wt · 2021-03-02T17:24:15Z

What is the point of split devices? Is it so that a different device node is used for the capture dev and the output dev?

umlaeute · 2021-03-02T21:01:41Z

@wt yes exactly. see also #322

vicamo · 2021-03-16T18:15:55Z

Update a few more works on this. Rebased to current devel branch HEAD and picked some useful commits from https://github.com/umlaeute/v4l2loopback/tree/feature/split-device. I have no intention to support any form of joined device as that's just impossible for vb2 framework, and it has bring all kinds problems/limitations to me. If that's unacceptable, then maybe you'd like to close this PR directly.

It's also based on the first commit of #414 that updates clang-formated source, and I think I'll need #415 to correctly handle frame sizes/intervals queries.

vicamo · 2021-03-17T16:33:59Z

merge fix v4l2 compliance errors #415
drop VIDIOCGMBUF support

When either vfh->ctrl_handler or vfd->ctrl_handler is non-null, it will shadow vidioc_queryctl. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

Fix incorrect format introduced in umlaeute#416 commit a572a4f ("report video-devices allocated at module-load time via sysfs"). Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

Converting function v4l2loopback_cd2dev and v4l2loopback_getdevice searches v4l2loopback_index_idr with device number retrieved from video_device drvdata, but this could be time consuming and creates unecessary complexity. And since we may need to retrieve v4l2_loopback_device address from callbacks shared between capture and output devices, this patch reuses video_device drvdata instead of struct member offsets. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

Since we have already instantiated an output video device, use its `num` field instead. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

These ioctls are only called as output role. Omit output/capture checks. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

vicamo · 2021-03-19T21:05:37Z

Rebased on For upstream/devel fixes #418
Remove exclusive_caps/announce_all_caps code
Pick fixes from feature/split-device branch
Return consistent data from enum_framesizes/enum_frameintervals as doc says so
Remove vidioc_s_fmt_overlay/vidioc_g_fmt_overlay/vidioc_querystd from output ioctl; they're capture-only,

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

According to V4L2 documentation: Applications can assume that the enumeration data does not change without any interaction from the application itself. This means that the enumeration data is consistent if the application does not perform any other ioctl calls while it runs the frame size enumeration. This change always returns all supported formats. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

umlaeute · 2021-03-22T08:02:37Z

* Rebased on #418

your commit style is great (i wish everybody would commit like this).

however, if possible please don't ~~rebase~~: instead use merge and cherry-pick.
it makes it much easier for me to review your changes.

vicamo · 2021-03-23T03:37:38Z

* Rebased on #418
your commit style is great (i wish everybody would commit like this).

however, if possible please don't ~~rebase~~: instead use merge and cherry-pick.
it makes it much easier for me to review your changes.

Hi, sorry for taking your extra time. Like what you have seen here, I'm picky at my own commits and always want to make them compact and self-explaining as possible. However this is still not a complete PR, therefore I mark it as WIP/RFC, and it has many aspects to be polished, re-organized, even re-written. So applies to the git commits, they just cannot meet my own expect yet, and many more works remain to be done at odd moments. I think I will do one more rebase when it's full ready for your review. Thank you.

alesya-h · 2022-05-22T08:51:10Z

@vicamo do you think this work will ever be complete? It blocks the release (see #398), it's been over a year since last comment on this thread, and it's been almost 5 months since the last commit in wip/split-device branch in your repo, so I assume development is kinda halted.

vicamo · 2022-05-22T17:56:02Z

@vicamo do you think this work will ever be complete? It blocks the release (see #398), it's been over a year since last comment on this thread, and it's been almost 5 months since the last commit in wip/split-device branch in your repo, so I assume development is kinda halted.

Yes, it's mostly halted as I'm more or less looking forward to pipewire/libcamera to serve as a similar role.

normanr · 2023-03-13T21:00:18Z

fwiw, it looks like https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/707 tracks the work for pipewire. It sounds like userspace is ~mostly working, but requires applications to update to support it.

wt · 2023-03-14T17:32:35Z

I don't have the time or inclination to continue this. the pipewire solution seems to be the way of the future. wt

…

On Mon, Mar 13, 2023 at 2:00 PM Norman Rasmussen ***@***.***> wrote: fwiw, it looks like https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/707 tracks the work for pipewire. It sounds like userspace is ~mostly working, but requires applications to update to support it. — Reply to this email directly, view it on GitHub <#359 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFT7ZRICTF4T42AWMBAGDW36DG5ANCNFSM4SSI2FSQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- You're More Amazing Than You Think! ymatyt.com

vicamo marked this pull request as draft October 15, 2020 17:23

umlaeute linked an issue Oct 15, 2020 that may be closed by this pull request

Add support for split output/capture devices. #322

Open

umlaeute added this to To do in upstream to kernel Oct 16, 2020

umlaeute moved this from To do to In Progress in upstream to kernel Oct 16, 2020

vicamo force-pushed the for-upstream/split-device branch 3 times, most recently from 42b9346 to 2da512a Compare October 19, 2020 08:06

vicamo force-pushed the for-upstream/split-device branch 2 times, most recently from c8db769 to 014227c Compare October 23, 2020 16:24

umlaeute force-pushed the for-upstream/split-device branch from bfce2d7 to 0f16525 Compare January 27, 2021 19:15

umlaeute changed the base branch from master to develop January 27, 2021 20:00

This was referenced Feb 4, 2021

Document an API-way to test for v4l2loopback module presence #397

Open

Last version has been released 10 months ago #398

Closed

umlaeute added this to the release 0.13 milestone Feb 5, 2021

vicamo force-pushed the for-upstream/split-device branch from 13ce229 to 447d702 Compare March 16, 2021 18:03

vicamo force-pushed the for-upstream/split-device branch from 447d702 to ac6439b Compare March 17, 2021 16:32

umlaeute mentioned this pull request Mar 18, 2021

[WIP] alternative split devices #417

Closed

vicamo added 2 commits March 20, 2021 03:07

remove V4L2 controls op handlers

799c5f3

When either vfh->ctrl_handler or vfd->ctrl_handler is non-null, it will shadow vidioc_queryctl. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

clang-format: follow-up

8a5b2da

Fix incorrect format introduced in umlaeute#416 commit a572a4f ("report video-devices allocated at module-load time via sysfs"). Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

vicamo added 8 commits March 20, 2021 04:10

cache vdev for latter uses

f843699

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

embed struct video_device into v4l2_loopback_device

d26166e

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

move v4l2_loopback_device->vdev into v4l2_loopback_device->capture

e11e83a

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

move all vdev init procedures into init_vdev

e6e1598

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

create a draft output video_device

9a75ec5

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

remove dev->output_nr

17fa487

Since we have already instantiated an output video device, use its `num` field instead. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

output: accommodate output specific ioctls

8738621

These ioctls are only called as output role. Omit output/capture checks. Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

vicamo force-pushed the for-upstream/split-device branch from ac6439b to 2daef4f Compare March 19, 2021 20:52

vicamo added 7 commits March 20, 2021 05:06

fill capabilities based on vdev->device_caps as possible

03a69b4

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

output: add ioctl handlers

09eb376

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

output: re-implement with videobuf2 framework

2d90d74

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

capture: init vb2_queue

07315b4

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

capture: use fops callback from videobuf2

dee0ad3

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

[wip] reimplement capture entity using videobuf2

a11fa3f

Signed-off-by: You-Sheng Yang <vicamo@gmail.com>

vicamo force-pushed the for-upstream/split-device branch from 2daef4f to a11fa3f Compare March 19, 2021 21:07

vicamo mentioned this pull request Dec 29, 2021

OBS Virtual Cam doesn't activate (Ubuntu 20.04) #404

Open

RytoEX mentioned this pull request Jan 22, 2022

Virtual Camera will start one time and will not restart until reboot obsproject/obs-studio#4808

Closed

BenBE mentioned this pull request Aug 16, 2022

New Feature Release? #500

Closed

polarathene mentioned this pull request Oct 30, 2022

v4l2loopback stops working after first close #504

Closed

umlaeute modified the milestones: release 0.13, release 0.14 May 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[rfc/wip] split device #359

[rfc/wip] split device #359

vicamo commented Oct 15, 2020 •

edited by umlaeute

umlaeute commented Oct 19, 2020

janakelarsson commented Jan 13, 2021

vicamo commented Jan 15, 2021

janakelarsson commented Jan 15, 2021

umlaeute commented Jan 15, 2021

janakelarsson commented Jan 15, 2021

janakelarsson commented Jan 19, 2021

wt commented Mar 2, 2021

umlaeute commented Mar 2, 2021

vicamo commented Mar 16, 2021 •

edited

vicamo commented Mar 17, 2021

vicamo commented Mar 19, 2021

umlaeute commented Mar 22, 2021

vicamo commented Mar 23, 2021

alesya-h commented May 22, 2022

vicamo commented May 22, 2022

normanr commented Mar 13, 2023

wt commented Mar 14, 2023 via email

[rfc/wip] split device #359

Are you sure you want to change the base?

[rfc/wip] split device #359

Conversation

vicamo commented Oct 15, 2020 • edited by umlaeute

umlaeute commented Oct 19, 2020

janakelarsson commented Jan 13, 2021

vicamo commented Jan 15, 2021

janakelarsson commented Jan 15, 2021

umlaeute commented Jan 15, 2021

janakelarsson commented Jan 15, 2021

janakelarsson commented Jan 19, 2021

wt commented Mar 2, 2021

umlaeute commented Mar 2, 2021

vicamo commented Mar 16, 2021 • edited

vicamo commented Mar 17, 2021

vicamo commented Mar 19, 2021

umlaeute commented Mar 22, 2021

vicamo commented Mar 23, 2021

alesya-h commented May 22, 2022

vicamo commented May 22, 2022

normanr commented Mar 13, 2023

wt commented Mar 14, 2023 via email

vicamo commented Oct 15, 2020 •

edited by umlaeute

vicamo commented Mar 16, 2021 •

edited