Module video4linux #249

0ry0n · 2022-01-20T12:53:04Z

I thought about introducing a module for v4l support in order to send the video payload to a /dev/video* device created via v4l2loopback.

This way you would get the video stream without using the "heavy" gstreamer, useful when you need the video stream locally. Example: (raspberry: neolink -> NVR software).

Unfortunately I don't know rust :(. I tried to implement it trying to follow the structure of the rtsp module, but there is some problem.

QuantumEntangledAndy

I've made some general comments if you could tell more about your specific issues I could give you some more specific comments

src/v4l/mod.rs

src/config.rs

src/v4l/mod.rs

src/v4l/v4lt.rs

QuantumEntangledAndy · 2022-01-21T01:25:40Z

We are probably also going to want a bunch of #[cfg(target_os = "linux"]] so that this code is only avaliable on linux. Read up on how to do that here

0ry0n · 2022-01-21T15:00:25Z

Ok, I have corrected these problems.
Now the problem is that I don't really know how to properly store device and stream in struct V4ltOutputs.
I relied on the example given in the libv4l module where device and stream are & mut.
I tried to implement them this way but the compiler gives me lifecicle problems and I have no idea how to fix them.

Thanks for your help!

QuantumEntangledAndy · 2022-01-21T23:25:34Z

I recommend you look up about rust ownership and lifetime it is a key component of the language. Once you get your head around it you can start to see its advantages and also how to work around things like your current issue.

& is a reference to data somewhere else. It cannot be reliably stored as it doesn't own the data. (There are ways to store it temporarily if the compiler can work out that the place it is being stored will be gone before the original data does, but in your case that isn't true)

Have you tried just storing them as Stream and Device no &? What happens when you do that? This will give your stuct ownership of the data.

QuantumEntangledAndy · 2022-01-21T23:31:57Z

src/v4l/mod.rs

+                        .add_stream()
+                        .unwrap();
+                    let main_camera = arc_cam.clone();
+                    s.spawn(move |_| camera_loop(&*main_camera, Stream::Main, &mut outputs, true));


Remove your &* this will deference then reference the variable (then your compiler will deterrence it again because the type of the function sig is Device not &Device. Also try to give the stream without an & too. So that your structure can take ownership but I'm not sure how v4l.add_stream works. Check the doc to see if it returns a reference or not. If it returns a reference we have to work out something different.

QuantumEntangledAndy · 2022-01-21T23:37:16Z

src/v4l/v4lt.rs

+
+pub(crate) struct V4ltOutputs<'a> {
+    device: Device,
+    stream: &'a mut MmapStream<'a>,


First I think we need to remove the &mut at the front. You seem to have ownership of it at this point.

However I suspect that there might be another issue in that the MmapStream has a lifetime parameter. This usually means that there is a reference in it that points to data else where. The compiler needs to be certain that the lifetime of 'a does not die. Which it cannot be certain of because of it running on a different thread. This will be an issue.

QuantumEntangledAndy · 2022-01-22T02:55:15Z

Have a look at the PR I made against your branch. 0ry0n#1

thirtythreeforty

Very interesting feature! Thank you for implementing it and thanks @QuantumEntangledAndy for the initial code review. I hadn't even considered doing something like this but I think it is a good idea.

A few points of feedback. I think the largest one is the code duplication in the wrapper module, and perhaps also (bikeshedding?) the name.

thirtythreeforty · 2022-02-24T20:04:06Z

src/v4l/v4lt.rs

+        // After we have created the device stream we cannot
+        // edit the height/width etc
+        // So first we pull packets from the camera until we have
+        // enough data to setup the height etc


Is there a way you can query metadata without parsing the video itself? There might be a camera Xml you can fetch that will tell you what its current settings are. (I've seen something similar but I can't recall if it's available or active settings.)

This data is also available in

<StreamInfoList> xml (msg id: 146)

<Compression> xml (msg id: 56)

Initial login (msg id 1)

But this MediaInfo packet was the easiest as the StreamInfoList and Compression aren't currently programmed in

thirtythreeforty · 2022-02-24T20:06:19Z

src/v4l/mod.rs

@@ -0,0 +1,270 @@
+///
+/// # Neolink V4l


This file appears to be about 95% copy-paste from the default stream command. The camera drive and setup loops are the same. Can we break those out into a common module and pass a "sink" trait to that module? I think the Gstreamer module is already kinda structured that way, so it would be an easy change to the existing code.

thirtythreeforty · 2022-02-24T20:07:25Z

src/v4l/mod.rs

+/// # Neolink V4l
+///
+/// This module serves the v4l streams for the
+/// `neolink v4l` subcommand


I object slightly to v4l as a name. Presumably Windows users would like something like this too and a hypothetical implementation using native Windows APIs would not be named v4l. What's a good name for the feature, not the mechanism?

Just as a reference: Macos has CoreMediaIO and windows has DirectShow for virtual camera. Neither of which seem to have any high level crates, it's just sys level FFI stuff at the moment, so we'd have to roll it ourselves.

We could perhaps get similar functionality though with just a fifo exposing the current streams.

I do kinda agree that the name should be generic rather than v4l (video 4 linux). Maybe virtual_camera?

Something pithy. virtcam or native?

👍 virtcam it is!

thirtythreeforty · 2022-02-24T20:10:04Z

src/config.rs

@@ -73,6 +79,26 @@ pub(crate) struct CameraConfig {
    #[validate(range(min = 0, max = 31, message = "Invalid channel", code = "channel_id"))]
    #[serde(default = "default_channel_id")]
    pub(crate) channel_id: u8,
+
+    #[cfg(target_os = "linux")]
+    // Maximum number of devices allowed by v4l2loopback (8)


It doesn't look like this limit is actually a hard limit, since it's possible to create additional devices:

umlaeute/v4l2loopback#367

Can you move this check to runtime, and fail with a nice error message if the user tries to create a device with a number that the kernel turns out not to like?

thirtythreeforty · 2022-02-24T20:13:38Z

And one last thing, don't forget to add the command to the readme under "additional commands"!

QuantumEntangledAndy reviewed Jan 21, 2022

View reviewed changes

thirtythreeforty requested changes Feb 24, 2022

View reviewed changes

0ry0n closed this Jan 15, 2023

0ry0n force-pushed the master branch from ff1a653 to 7087b34 Compare January 15, 2023 19:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Module video4linux #249

Module video4linux #249

0ry0n commented Jan 20, 2022

QuantumEntangledAndy left a comment

QuantumEntangledAndy commented Jan 21, 2022

0ry0n commented Jan 21, 2022

QuantumEntangledAndy commented Jan 21, 2022 •

edited

Loading

QuantumEntangledAndy Jan 21, 2022

QuantumEntangledAndy Jan 21, 2022 •

edited

Loading

QuantumEntangledAndy commented Jan 22, 2022

thirtythreeforty left a comment

thirtythreeforty Feb 24, 2022

QuantumEntangledAndy Feb 26, 2022

thirtythreeforty Feb 24, 2022

thirtythreeforty Feb 24, 2022

QuantumEntangledAndy Feb 26, 2022 •

edited

Loading

thirtythreeforty Mar 2, 2022

0ry0n Mar 15, 2022

thirtythreeforty Mar 15, 2022

thirtythreeforty Feb 24, 2022

thirtythreeforty commented Feb 24, 2022

Module video4linux #249

Module video4linux #249

Conversation

0ry0n commented Jan 20, 2022

QuantumEntangledAndy left a comment

Choose a reason for hiding this comment

QuantumEntangledAndy commented Jan 21, 2022

0ry0n commented Jan 21, 2022

QuantumEntangledAndy commented Jan 21, 2022 • edited Loading

Choose a reason for hiding this comment

QuantumEntangledAndy Jan 21, 2022 • edited Loading

Choose a reason for hiding this comment

QuantumEntangledAndy commented Jan 22, 2022

thirtythreeforty left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

QuantumEntangledAndy Feb 26, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thirtythreeforty commented Feb 24, 2022

QuantumEntangledAndy commented Jan 21, 2022 •

edited

Loading

QuantumEntangledAndy Jan 21, 2022 •

edited

Loading

QuantumEntangledAndy Feb 26, 2022 •

edited

Loading