Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add zero-copy NVMM support with nvbufsurface (JetPack 5) #204

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

poett1
Copy link

@poett1 poett1 commented Apr 24, 2024

Just had some time today/yesterday and updated to the new API changes for nvbufsurface on JP5.
I checked this with IMX219 (raw) using Argus and See3CAM_CU135 (raw/jpeg) on JP5.1.2 (Orin NX). I hope someone can check this again with Jetpack 4.6. Currently, I do not have any devices running Jetpack below 5.1.2.

There are two main changes:

1. nvv4l2camerasrc:
I updated the code to use nvv4l2camerasrc with v4l2 cameras (that do not support Argus) if input_codec=RAW on Jetson Devices.
This will output a surface with NVBUF_LAYOUT_PITCH, which is transformed to NVBUF_LAYOUT_BLOCK_LINEAR in gstBufferManager::Enqueue (thus we need to link against nvbufsurftransform).
Also, it seems like eglFrame.planeCount=1 for this source. I commented out the check for this in gstBufferManager::Dequeue to disabled warnings. If you have some other ideas for this, feel free to change it, but it seems to work fine.
Sometimes nvv4l2camerasrc outputs bad frames (mostly green with a bar of snow). This also happens with gst-launch-1.0 on my Jetson. Reconnecting the USB camera again seems to solve it.

This can be checked with:
video-viewer /dev/video1 --input-codec=raw which will create the pipeline:
nvv4l2camerasrc device=/dev/video1 do-timestamp=true ! video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1280, height=(int)720, framerate=30/1 ! appsink name=mysink sync=false

2. nvvidconv:
If ENABLE_NVMM=ON we will always use nvvidconv to rotate the video now, even if the source does not output NVMM. nvvidconv will output NVMM to appsink.

This can be checked with:
video-viewer /dev/video1 --input-flip=rotate-180
which will create the pipeline:
v4l2src device=/dev/video1 do-timestamp=true ! image/jpeg, width=(int)1280, height=(int)720, framerate=60/1 ! jpegdec name=decoder ! video/x-raw ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM) ! appsink name=mysink sync=false

And with CSI Cameras:
video-viewer csi://0 --input-flip=rotate-180
the pipeline will be:
nvarguscamerasrc sensor-id=0 ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, framerate=30/1, format=(string)NV12 ! nvvidconv flip-method=0 ! video/x-raw(memory:NVMM) ! appsink name=mysink

I will be happy to help If any additional changes are necessary to get this into main.

@dusty-nv
Copy link
Owner

Thank you @poett1! I appreciate this a lot! I will need some time to test this on JetPack 4, 5, and 6 first, or #ifdef it for different versions of JetPack. Otherwise users may randomly have camera issues (which is why over time I have just disabled the NVMM because it was more trouble to maintain than it was worth)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants