Issues with 2 camera's on M8P with CM4 & CB1 - crowsnest#109

[mryel00]

This Discussion should be a place to discuss about mainsail-crew/crowsnest#109.
Feel free to talk about this issue or ask for help about possible brought up solutions.

The mainsail-crew will not provide any help, if this is a hardware issue (as we currently think) and we will only provide the platform for the discussion.
We would be grateful if our community can help each other to find a good solution even for the non-tech-savvy users.

One found maybe possible solution and explanation of that issue.

[mryel00]

We are going to run a bunch of comprehensive tests tomorrow morning to get empirical results when using the same cameras with different setups. Will let you know what we find.

@looxonline Sry to bother you, but did you guys already run those tests? I'm very interested in the results.

[mryel00]

Amazing work 😄
These results are pretty interesting, especially that the Pi3 is reporting some weird stuff. Maybe some kernel modifications?

Counter intuitively, if mainsail team adds the option to run ustreamer in YUYV mode, we will be able to stream two cams in parallel even though YUYV is not compressed and MJPG is.

Overall we already support that with custom_flags. Someone can add --format=YUYV and it will override the "forced" MJPG format.

Changing from MJPEG to YUYV and applying the kernel module quirk dramatically reduces the bus bandwidth use per camera and allows more than one on a high speed bus.

So it's enabling YUYV and the quirk, so there isn't an easy way to fix it directly during runtime of crowsnest, if I understood that correctly. We shouldn't reload the uvc kernel module on start of crowsnest, as it might have some side effects for some specific setups.
We will discuss it internally if we can apply that quirk during install on the CB1 & CM4 and then set force --format=YUYV for those 2.

[looxonline]

Yes it must be both YUYV and the quirk. The quirk only applies to uncompressed streams unfortunately. In the other post I showed a kernel driver mod that would cause it to apply to compressed streams but that is far beyond the scope of most users.

I think it would be a good idea to force the quirk regardless of whether those modules are being used. Since the quirk only applies to uncompressed streams and almost all users use MJPEG it would have no effect on them and for those using YUYV, it will only have a positive effect since it does not under estimate the needed bandwidth, it just calculates it correctly.

By doing this, you can reduce the problem down to only needing to force YUYV on those modules and not have to worry about when or if to apply the quirk.

[mryel00]

I think it would be a good idea to force the quirk regardless of whether those modules are being used.

We will discuss it internally, on what scope makes sense.
I hesitate a bit on applying it to every machine, even if it doesn't harm, as some users might not like it, if we apply it without their consent and a popup to ask might confuse others.

Anyway it won't come before the next big update, as we are currently rewriting everything of crowsnest and only apply a few last bug fixes atm to the current version.

Also it's good to know that it's actually more like a "bug" with the "normal" Raspberry Pis that they report wrong stuff, as this issue then might occur on other machines too. Actually another good reason to apply it overall, ig...

[mryel00]

Another small detail that I forgot in my previous response:

Ustreamer can be a PITA and often tries to max out the FPS in spite of what is configured. The image below shows how it jumps to something different to what is configured, almost maxing out what it thinks the bus can handle for the given res. This may cause frustration as users try to drop the FPS but find that it makes no impact to how many cameras they can run.

Ustreamer sets the fps to what the camera is capable of. The camera can provide different encoded streams at specific resolutions with specific fps. Those can be shown with v4l2-ctl -d <device> --list-ctrls-menus. Those are fixed and nothing ustreamer can change. Those calues are different from cam to cam. A small snippet as an example of the output:

[0]: 'MJPG' (Motion-JPEG, compressed):
        Size: Discrete 2560x1440
                Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 1920x1080
                Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 640x480
                Interval: Discrete 0.017s (60.000 fps)
....
        Size: Discrete 1600x1200
                Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 960x720
                Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 1280x960
                Interval: Discrete 0.033s (30.000 fps)
[1]: 'YUYV' (YUYV 4:2:2):
        Size: Discrete 2560x1440
                Interval: Discrete 0.500s (2.000 fps)
        Size: Discrete 1920x1080
                Interval: Discrete 0.500s (2.000 fps)
        Size: Discrete 640x480
                Interval: Discrete 0.033s (30.000 fps)
....
        Size: Discrete 1600x1200
                Interval: Discrete 0.200s (5.000 fps)
        Size: Discrete 960x720
                Interval: Discrete 0.067s (15.000 fps)
        Size: Discrete 1280x960
                Interval: Discrete 0.200s (5.000 fps)

As you can see it differs from resolution to resolution what the cam can provide and it's different for the different encodings too. As YUYV isn't compressed you see as low as 2 fps. Those values shown, are the only combinations that will work, and you cannot enforce e.g. 1920x1080@20fps with MJPG (maybe with some kernel modifications, but this is out of scope of this topic)

[looxonline]

The thing is that we had already confirmed that the FPS that we wanted were available at the resolutions that we had selected using the v4l2 compatibility list. It was really frustrating because it kept jumping to another even when the one we wanted was listed and should have worked.

[mryel00]

As a small summary to the proposed solution:

• Enable following quirk for the Linux UVC Driver: https://stackoverflow.com/a/25619508 (To survive reboots look at the answer below that)
• Use custom_flags: --format=YUYV for mode: ustreamer and custom_flags: --camera-format=YUYV for mode: camera-streamer inside your crowsnest.conf

After those two steps, you should be able to use 2 cams on your CB1 and CM4.
Thanks again for providing a solution, the long investigation and the good tests of @looxonline and his team.

If someone got questions or need help, feel free to ask in this discussion 😄

P.S. The FPS with YUYV can be a lot lower than with MJPG. This differs from cam to cam.

[dragz]

Hi, and thanks for all the work you all put into this. I just tested the proposed solution and have made the following observations:

(0. I already have a working solution with two cams using camera-streamer on a M8P board with a CM4. Resolution has to set at 640x480 and fps to 10 on both cams.)

1. The suggested config does not improve on the situation for me. Switching to ustreamer with --format=YUYV and setting the uvcvideo param quirks=640 or 128 still does not let me increase the resolution or fps.
2. The suggested solution creates a lot more load on the cpu, 30-50%, just running the setup without printing. Using camera-streamer the cpu-load stays below 5%.

I´d be happy to do more testing if there are other suggestions to try out.

[mryel00]

If I understood correctly, only UVC_QUIRK_FIX_BANDWIDTH should be need to be applied, therefore only 128.
The FPS value should not increase that much, as I already noted above. For the resolution it should work ig. Some of the crew will do testing internally soon, then I will have own results on this solution. I'm currently solely relying on the messages of looxonline.

As a note about camera-streamer, you can try to change the format there too: custom_flags: --camera-format=YUYV
I somehow forgot that the CM4 got camera-streamer support. I added it to the above message too.

[dragz]

Still no improvement on my setup. I can confirm that the custom_flags setting works for camera-streamer, but I cannot run both cameras at a resolution higher than 640x480. Anyway, I came to realize that as my second cam only watches the spool I don't need full resolution on it. I adjusted it down to 320x240 and fps=1 and the primary cam up to 1280x1024 and fps=20. It works both with and without the quirks and custom_flags settings.

Again, thanks for all the effort you put into tracking this down.

[mryel00]

I got some test results from a crew member. He tried the proposed fix, but it seems like after applying it, he isn't able to use YUYV encoding anymore and is getting CAP: Can't start capturing: Input/output error or Mainloop select() error: Success when using custom_flags: --format=YUYV, any idea @looxonline? Did you come across something similar maybe?

Furthermore I discovered that ustreamer isn't working on the screenshots of @looxonline. Did you look at the streams when testing the quirk, or only at the terminal data?

[dragz]

I had to use the "Adaptive MJPEG streamer" when testing ustreamer.

[mryel00]

Fyi, these 2 errors appear without opening up any stream and force ustreamer to restart every second.