Allow to set additional parameters of the Raspberry Pi Camera

[aler9]

Describe the feature

The Raspberry Pi Camera has a lot of parameters that can be tuned and should be added to the server configuration.

List:

libcamera-vid --help

List of parameters to take into consideration:

• --awb
• --denoise cdn_off
• --mode 1332:990:10
• --level 4.2
• --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx477.json
• --mode
• --post-process-file

[cedricve]

Agreed +1, currently we have a work around like in this repo -> https://github.com/kerberos-io/camera-to-rtsp

[olokos]

Agreed +1, currently we have a work around like in this repo -> https://github.com/kerberos-io/camera-to-rtsp

I have to say that this approach also works amazingly well, allowing the use of all possible libcamera-vid/still parameters!

Been using it with my HQ Camera/IMX477, I had to build both libcamera+libcamera-apps from source to use it's full capabilities (like 1332x990@90FPS, up to 120FPS is possible).

Having native support for it with rtsp-simple-server would've been amazing though.

I've been thinking of using the power of HEVC/H265 and possibly also disable the camera during night time, since the sensor doesn't see anything anyway, unless I mount some massive IR blasters and removing the IR filter.

Regarding the night time I was thinking of doing some cronjob that would run hourly:

1. is the camera running?
2. get time of local sunset
3. if local sunset is in 2 hours or less -> turn off the stream
4. if local sunrise is in 1 hour or less -> turn on the stream.

Above scheduling of camera + H265 support, on top of being able to use all of the parameters of the camera, as originally implemented, would make rtsp-simple-server so good there would be little point of using the camera outside of rtsp-simple-server, but those are just few of my ideas.

Above all of the things I mentioned, I wanted to say Thank You very much aler9 for Your support and amazing work, that everybody could use, I highly appreciate the work and effort put into this! <3

[aler9]

Agreed +1, currently we have a work around like in this repo -> https://github.com/kerberos-io/camera-to-rtsp

The problem of that approach is that is slow compared to the one that can be obtained by interacting directly with libcamera, and that's why i wanted to integrate it into the server. Anyway, since rtsp-simple-server is now able to interact with libcamera, adding support for additional parameters is actually really simple, it's enough to take all the set() functions here:

https://github.com/raspberrypi/libcamera-apps/blob/e1beb4512178f5a229001b1ecc378a89bc9757c0/core/libcamera_app.cpp#L422

and paste them into the server, here:

https://github.com/aler9/rtsp-simple-server/blob/27fb9fdb7ea53f8d13a0fda2a6a59b361f87a4c3/internal/rpicamera/exe/camera.cpp#L191

I don't know whey i'll got the time to do it, but in the meanwhile, anyone is free to do the work and submit a patch.

H265 support

Raspberry Pis at the moment aren't equipped with a H265 hardware encoder:

https://forums.raspberrypi.com/viewtopic.php?t=243873

And i don't recommend doing H265 software encoding since it would consume most of the CPU.

[olokos]

Thank You, I've just watched a video about software encoding H265 performance and I see even with powerful Desktop CPU the performance is horrible, so there's no point going that path.

Regarding the camera parameters, there are some more parameters not listed in those set().
namely:
--denoise cdn_off
--mode 1332:990:10
--framerate 100
--width 1332
--height 990
--level 4.2
--tuning-file /usr/share/libcamera/ipa/raspberrypi/imx477.json

Which I believe are very useful, so I'm leaving them here for the future.

[cedricve]

Agreed +1, currently we have a work around like in this repo -> https://github.com/kerberos-io/camera-to-rtsp

The problem of that approach is that is slow compared to the one that can be obtained by interacting directly with libcamera, and that's why i wanted to integrate it into the server. Anyway, since rtsp-simple-server is now able to interact with libcamera, adding support for additional parameters is actually really simple, it's enough to take all the set() functions here:

https://github.com/raspberrypi/libcamera-apps/blob/e1beb4512178f5a229001b1ecc378a89bc9757c0/core/libcamera_app.cpp#L422

and paste them into the server, here:

https://github.com/aler9/rtsp-simple-server/blob/27fb9fdb7ea53f8d13a0fda2a6a59b361f87a4c3/internal/rpicamera/exe/camera.cpp#L191

I don't know whey i'll got the time to do it, but in the meanwhile, anyone is free to do the work and submit a patch.

H265 support

Raspberry Pis at the moment aren't equipped with a H265 hardware encoder:

https://forums.raspberrypi.com/viewtopic.php?t=243873

And i don't recommend doing H265 software encoding since it would consume most of the CPU.

Agreed! What still would be great to have, is a single container running with injecting the appropriate devices into the container.

As far I've understood the code to interface libcamera is not yet released inside the latest, docker image.

[giano574]

Thank You, I've just watched a video about software encoding H265 performance and I see even with powerful Desktop CPU the performance is horrible, so there's no point going that path.

Regarding the camera parameters, there are some more parameters not listed in those set(). namely: --denoise cdn_off --mode 1332:990:10 --framerate 100 --width 1332 --height 990 --level 4.2 --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx477.json

Which I believe are very useful, so I'm leaving them here for the future.

+1
I need to be able to specify --mode for my use case.
--post-process-file as well.

[aler9]

The following parameters have been added:

• rpiCameraHFlip
• rpiCameraVFlip
• rpiCameraBrightness
• rpiCameraContrast
• rpiCameraSaturation
• rpiCameraSharpness
• rpiCameraExposure
• rpiCameraAWB
• rpiCameraDenoise
• rpiCameraShutter
• rpiCameraMetering
• rpiCameraGain
• rpiCameraEV
• rpiCameraROI
• rpiCameraTuningFile

The following parameters have been left out of the PR:

• -- mode: it isn't used inside libcamera-vid (see here), mode is inferred automatically from width, height and fps
• --post-process-file: it controls a complex, pure-software process written inside libcamera-vid, subjected to regular updates and with external dependencies (opencv). I can't copy and paste libcamera-vid files and i can't add opencv to the server dependencies.

[reyoda]

• -- mode: it isn't used inside libcamera-vid (see here), mode is inferred automatically from width, height and fps

--mode is actually used in libcamera-vid.
have_raw_stream is set to true when options_->mode.bit_depth is set which happens in the Mode constructor when called in options.cpp:238.

It is useful for choosing a higher resolution sensor mode in combination with a lower resolution output. An example libcamera-vid command could be:

libcamera-vid --mode 3840:2160 --width 1920 --height 1080 --framerate 18

This gives (for some cameras at least) a higher quality image than the binned alternative.

Note that the maximum framerate is determined by the selected sensor mode.

[aler9]

@reyoda thanks for clarifying this, evidently i didn't understood the code enough, i'll give another try.

[aler9]

rpiCameraMode has been implemented

[aler9]

added in v0.20.1

[FantomeP]

Hello,
I tried implementing a NoIR camera v2 on a pi3a+ into a birdbox, and i'm quite pleased with mediamtx so far. I just would love getting a greyworld AWB on my stream, but i can't get it working.
#1181 refered to this problem already and it was closed like this thread => is there a possibility to get this greyworld parameter for rpiCameraAWB ? I've tried to stay in auto and add the auto_awb_is_greyworld=1 into my config.txt, but with no result...
Setting saturation to 0 is supposed to convert to greyscale too, it's working with Libcamera-vid but not with mediamtx.yml.
My stream is a splendid pink frame in daylight, and a solid blue during night without this feature... So if there's anyone who could get it work, i'd be glad having some help ! :)
Thanks

[github-actions]

This issue is being locked automatically because it has been closed for more than 6 months.
Please open a new issue in case you encounter a similar problem.