Add zero-copy NVMM support with nvbufsurface (JetPack 5) #204
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Just had some time today/yesterday and updated to the new API changes for nvbufsurface on JP5.
I checked this with IMX219 (raw) using Argus and See3CAM_CU135 (raw/jpeg) on JP5.1.2 (Orin NX). I hope someone can check this again with Jetpack 4.6. Currently, I do not have any devices running Jetpack below 5.1.2.
There are two main changes:
1. nvv4l2camerasrc:
I updated the code to use
nvv4l2camerasrc
with v4l2 cameras (that do not support Argus) ifinput_codec=RAW
on Jetson Devices.This will output a surface with NVBUF_LAYOUT_PITCH, which is transformed to NVBUF_LAYOUT_BLOCK_LINEAR in
gstBufferManager::Enqueue
(thus we need to link against nvbufsurftransform).Also, it seems like
eglFrame.planeCount=1
for this source. I commented out the check for this ingstBufferManager::Dequeue
to disabled warnings. If you have some other ideas for this, feel free to change it, but it seems to work fine.Sometimes
nvv4l2camerasrc
outputs bad frames (mostly green with a bar of snow). This also happens with gst-launch-1.0 on my Jetson. Reconnecting the USB camera again seems to solve it.This can be checked with:
video-viewer /dev/video1 --input-codec=raw
which will create the pipeline:nvv4l2camerasrc device=/dev/video1 do-timestamp=true ! video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1280, height=(int)720, framerate=30/1 ! appsink name=mysink sync=false
2. nvvidconv:
If
ENABLE_NVMM=ON
we will always usenvvidconv
to rotate the video now, even if the source does not output NVMM.nvvidconv
will output NVMM to appsink.This can be checked with:
video-viewer /dev/video1 --input-flip=rotate-180
which will create the pipeline:
v4l2src device=/dev/video1 do-timestamp=true ! image/jpeg, width=(int)1280, height=(int)720, framerate=60/1 ! jpegdec name=decoder ! video/x-raw ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM) ! appsink name=mysink sync=false
And with CSI Cameras:
video-viewer csi://0 --input-flip=rotate-180
the pipeline will be:
nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, framerate=30/1, format=(string)NV12 ! nvvidconv flip-method=0 ! video/x-raw(memory:NVMM) ! appsink name=mysink
I will be happy to help If any additional changes are necessary to get this into main.