libav changes #606

naushir · 2023-11-21T11:52:34Z

No description provided.

Add a general codec options function that will allow other libav encoders to be selected correctly. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

Test that we can create a small mkv/mp4 file using libav + libx264 correctly. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

naushir · 2023-11-21T12:29:07Z

@popsUlfr can you test this change before I go ahead and merge it?

popsUlfr · 2023-11-21T13:17:45Z

Yes @naushir it works

$ libcamera-vid -t 10000 --width 1280 --height 720 --framerate 30 --tuning-file /usr/share/libcamera/ipa/rpi/pisp/imx708_wide.json --codec libav --libav-video-codec librav1e -o video.webm

[0:06:59.284435023] [2735]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+117-99b177b3
[0:06:59.311320165] [2739]  INFO RPI pisp.cpp:653 libpisp version v1.0.1 5652744a74d7 21-11-2023 (12:04:59)
[0:06:59.330620533] [2739]  INFO RPI pisp.cpp:1112 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media3 and ISP device /dev/media1 using PiSP variant BCM2712_C0
Made X/EGL preview window
Mode selection for 1280:720:12:P(30)
    SRGGB10_CSI2P,1536x864/120.135 - Score: 1100
    SRGGB10_CSI2P,2304x1296/56.0255 - Score: 1400
    SRGGB10_CSI2P,4608x2592/14.3536 - Score: 33592.8
Stream configuration adjusted
[0:06:59.763793388] [2735]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-YUV420 (1) 1536x864-BGGR16_PISP_COMP1
[0:06:59.763936468] [2739]  INFO RPI pisp.cpp:1396 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
Output #0, webm, to 'video.webm':
  Stream #0:0: Video: av1, yuv420p(tv, bt709), 1280x720, q=2-31, 30 fps, 30 tbr, 1000k tbn
Halting: reached timeout of 10000 milliseconds.

$ libcamera-vid -t 10000 --width 1280 --height 720 --framerate 30 --tuning-file /usr/share/libcamera/ipa/rpi/pisp/imx708_wide.json --codec libav --libav-video-codec libx265 -o video.ts

[0:08:38.098800626] [3166]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+117-99b177b3
[0:08:38.117548708] [3170]  INFO RPI pisp.cpp:653 libpisp version v1.0.1 5652744a74d7 21-11-2023 (12:04:59)
[0:08:38.137972166] [3170]  INFO RPI pisp.cpp:1112 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media3 and ISP device /dev/media1 using PiSP variant BCM2712_C0
Made X/EGL preview window
Mode selection for 1280:720:12:P(30)
    SRGGB10_CSI2P,1536x864/120.135 - Score: 1100
    SRGGB10_CSI2P,2304x1296/56.0255 - Score: 1400
    SRGGB10_CSI2P,4608x2592/14.3536 - Score: 33592.8
Stream configuration adjusted
[0:08:38.533476149] [3166]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-YUV420 (1) 1536x864-BGGR16_PISP_COMP1
[0:08:38.533575437] [3170]  INFO RPI pisp.cpp:1396 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
x265 [info]: HEVC encoder version 3.5+1-f0c1022b6
x265 [info]: build info [Linux][GCC 12.2.0][32 bit][noasm] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: NEON
x265 [info]: Main profile, Level-3.1 (Main tier)
x265 [warning]: No thread pool allocated, --wpp disabled
x265 [warning]: No thread pool allocated, --lookahead-slices disabled
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / none
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 3 / 30 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing deblock sao
Output #0, mpegts, to 'video.ts':
  Stream #0:0: Video: hevc, yuv420p(tv, bt709), 1280x720, q=2-31, 30 fps, 30 tbr, 1000k tbn
Halting: reached timeout of 10000 milliseconds.
x265 [info]: frame I:      2, Avg QP:29.11  kb/s: 1157.76
x265 [info]: frame P:     11, Avg QP:29.63  kb/s: 331.68
x265 [info]: frame B:     44, Avg QP:34.40  kb/s: 32.77
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 7.7% 0.0% 0.0% 30.8% 61.5%

encoded 57 frames in 17.56s (3.25 fps), 129.93 kb/s, Avg QP:33.29

$ libcamera-vid -t 10000 --width 1280 --height 720 --framerate 30 --tuning-file /usr/share/libcamera/ipa/rpi/pisp/imx708_wide.json --codec libav --libav-video-codec libvpx-vp9 -o video.webm

[0:10:27.433516700] [3304]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+117-99b177b3
[0:10:27.457001351] [3308]  INFO RPI pisp.cpp:653 libpisp version v1.0.1 5652744a74d7 21-11-2023 (12:04:59)
[0:10:27.473275037] [3308]  INFO RPI pisp.cpp:1112 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media3 and ISP device /dev/media1 using PiSP variant BCM2712_C0
Made X/EGL preview window
Mode selection for 1280:720:12:P(30)
    SRGGB10_CSI2P,1536x864/120.135 - Score: 1100
    SRGGB10_CSI2P,2304x1296/56.0255 - Score: 1400
    SRGGB10_CSI2P,4608x2592/14.3536 - Score: 33592.8
Stream configuration adjusted
[0:10:27.878748329] [3304]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-YUV420 (1) 1536x864-BGGR16_PISP_COMP1
[0:10:27.878832638] [3308]  INFO RPI pisp.cpp:1396 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
[libvpx-vp9 @ 0x5555dd2fe760] v1.12.0
[libvpx-vp9 @ 0x5555dd2fe760] Neither bitrate nor constrained quality specified, using default CRF of 32
Output #0, webm, to 'video.webm':
  Stream #0:0: Video: vp9, yuv420p(tv, bt709), 1280x720, q=2-31, 30 fps, 30 tbr, 1000k tbn
Halting: reached timeout of 10000 milliseconds.

$ libcamera-vid -t 10000 --width 1280 --height 720 --framerate 30 --tuning-file /usr/share/libcamera/ipa/rpi/pisp/imx708_wide.json --codec libav --libav-video-codec libx264 -o video.ts

[0:11:12.140223859] [3359]  INFO Camera camera_manager.cpp:284 libcamera v0.1.0+117-99b177b3
[0:11:12.159636272] [3363]  INFO RPI pisp.cpp:653 libpisp version v1.0.1 5652744a74d7 21-11-2023 (12:04:59)
[0:11:12.180447566] [3363]  INFO RPI pisp.cpp:1112 Registered camera /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a to CFE device /dev/media3 and ISP device /dev/media1 using PiSP variant BCM2712_C0
Made X/EGL preview window
Mode selection for 1280:720:12:P(30)
    SRGGB10_CSI2P,1536x864/120.135 - Score: 1100
    SRGGB10_CSI2P,2304x1296/56.0255 - Score: 1400
    SRGGB10_CSI2P,4608x2592/14.3536 - Score: 33592.8
Stream configuration adjusted
[0:11:12.573422348] [3359]  INFO Camera camera.cpp:1183 configuring streams: (0) 1280x720-YUV420 (1) 1536x864-BGGR16_PISP_COMP1
[0:11:12.573580966] [3363]  INFO RPI pisp.cpp:1396 Sensor: /base/axi/pcie@120000/rp1/i2c@80000/imx708@1a - Selected sensor format: 1536x864-SBGGR10_1X10 - Selected CFE format: 1536x864-PC1B
[libx264 @ 0x5555f45d6040] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x5555f45d6040] profile High, level 3.1, 4:2:0, 8-bit
Output #0, mpegts, to 'video.ts':
  Stream #0:0: Video: h264, yuv420p(tv, bt709), 1280x720, q=2-31, 30 fps, 30 tbr, 1000k tbn
Halting: reached timeout of 10000 milliseconds.
[libx264 @ 0x5555f45d6040] frame I:10    Avg QP:15.43  size: 53191
[libx264 @ 0x5555f45d6040] frame P:146   Avg QP:17.12  size: 17636
[libx264 @ 0x5555f45d6040] frame B:137   Avg QP:19.22  size:  2621
[libx264 @ 0x5555f45d6040] consecutive B-frames:  6.5% 93.5%
[libx264 @ 0x5555f45d6040] mb I  I16..4: 22.0% 67.3% 10.7%
[libx264 @ 0x5555f45d6040] mb P  I16..4: 16.3% 22.5%  0.1%  P16..4: 43.3%  0.0%  0.0%  0.0%  0.0%    skip:17.8%
[libx264 @ 0x5555f45d6040] mb B  I16..4:  0.4%  0.2%  0.0%  B16..8: 15.3%  0.0%  0.0%  direct:22.6%  skip:61.5%  L0:30.3% L1:66.6% BI: 3.1%
[libx264 @ 0x5555f45d6040] 8x8 transform intra:58.9% inter:34.2%
[libx264 @ 0x5555f45d6040] coded y,uvDC,uvAC intra: 76.5% 42.4% 11.4% inter: 10.4% 32.1% 0.7%
[libx264 @ 0x5555f45d6040] i16 v,h,dc,p: 12% 16% 40% 32%
[libx264 @ 0x5555f45d6040] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 11% 49%  4%  9%  4%  6%  4%  7%
[libx264 @ 0x5555f45d6040] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 12% 14% 28% 11%  9%  4%  6%  8%  8%
[libx264 @ 0x5555f45d6040] i8c dc,h,v,p: 52% 22% 18%  8%
[libx264 @ 0x5555f45d6040] kb/s:2840.39

naushir · 2023-11-21T13:21:35Z

Great thanks! Merging now.

axeljerabek · 2024-02-19T09:12:23Z

Please add the option to simply copy the raw h264 stream from the sensor without having to reencode it with CPU. I did not find any option like 'copy' for the '--libav-video-codec' option. WIthout this option, the raw h264 stream was simply transported to an rtmp:// server which then could record it.

It was possible to pickup the cam stream in h264_raw format and simply transfer it over a network with around 7-10 % of CPU usage. I use these streams then on a bigger server to record with motion detection. I can even pickup a FullHD h264 stream on a Raspberry Pi Zero 2W with this method. Reencoding on such a tiny CPU is not an option and will break the video streams while heating up the tiny Raspberries.

What worked before the upgrade:

rpicam-vid -t 0 --width 1920 --height 1080 --nopreview --exposure long --sharpness 1.1 --contrast 1.1 --brightness 0.2 --saturation 1.0 --awb auto --denoise auto
--level 4.2 --codec libav --libav-video-codec yuv420 --libav-audio --audio-source alsa --audio-device hw:2,0 --audio-channels 1 --audio-codec aac --audio-samplerate 48000 --audio-bitrate 128000 --av-sync 0
--libav-format flv -n --framerate 30 -b 5200000 --autofocus-mode manual --lens-position 0.25 --autofocus-window 0.25,0.25,0.5,0.5 --tuning-file /usr/share/libcamera/ipa/rpi/vc4/imx708.json
--vflip --hflip --inline --save-pts timestamp.pts -o "rtmp://192.168.178.25/pi/test/

libav_encoder: Allow other libav encoders to be selected.

efcf4b8

Add a general codec options function that will allow other libav encoders to be selected correctly. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

naushir mentioned this pull request Nov 21, 2023

libav_encoder: Allow other libav encoders to be selected. #602

Closed

naushir force-pushed the libav branch from 6ac9a87 to 8bc3ec3 Compare November 21, 2023 12:02

tests: Add a libav encoder test

a11ae59

Test that we can create a small mkv/mp4 file using libav + libx264 correctly. Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

naushir force-pushed the libav branch from 8bc3ec3 to a11ae59 Compare November 21, 2023 12:20

naushir merged commit 16647b4 into main Nov 21, 2023
10 checks passed

naushir deleted the libav branch November 21, 2023 13:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libav changes #606

libav changes #606

naushir commented Nov 21, 2023

naushir commented Nov 21, 2023

popsUlfr commented Nov 21, 2023

naushir commented Nov 21, 2023

axeljerabek commented Feb 19, 2024 •

edited

Loading

libav changes #606

libav changes #606

Conversation

naushir commented Nov 21, 2023

naushir commented Nov 21, 2023

popsUlfr commented Nov 21, 2023

naushir commented Nov 21, 2023

axeljerabek commented Feb 19, 2024 • edited Loading

axeljerabek commented Feb 19, 2024 •

edited

Loading