Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kernel 6 and gstreamer OpenGL is unusable slow? #117

Open
chrisiberg opened this issue Jul 1, 2024 · 25 comments
Open

Kernel 6 and gstreamer OpenGL is unusable slow? #117

chrisiberg opened this issue Jul 1, 2024 · 25 comments

Comments

@chrisiberg
Copy link

with Kernel 5.10 and Kernel 6 there is the same error.

DISPLAY=:0 glxinfo | grep "OpenGL renderer"
libGL error: MESA-LOADER: failed to open rockchip: /usr/lib/dri/rockchip_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri)
libGL error: failed to load driver: rockchip
libGL error: MESA-LOADER: failed to open rockchip: /usr/lib/dri/rockchip_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri)
libGL error: failed to load driver: rockchip
OpenGL renderer string: softpipe

when i use this gstreamer pipeline

gst-launch-1.0 v4l2src device=/dev/video0 ! glupload ! glcolorconvert ! glcolorscale ! video/x-raw(memory:GLMemory), format=RGBA, width=1920, height=1080 ! gldownload ! glimagesink

it's super fast with nearly no CPU usage with Kernel 5.10 but with Kernel 6 it is unusable slow?

Do you have an idea Jeffy?
@chrisiberg chrisiberg changed the title Kernel 6 and OpenGL is unusable slow? Kernel 6 and gstreamer OpenGL is unusable slow? Jul 2, 2024
@chrisiberg
Copy link
Author

downgrading to gstreamer 1.20.7 did not help.

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 2, 2024

if let me guess, there might be a memcpy() in v4l2src -> glupload, the v4l2src might provide non-dmabuf to glupload, which would cause a memcpy() when uploading userspace buf to texture in mali DDK.

you can try to "export GST_DEBUG=v4l2bufferpool:5" to check the logs.

and you can try to workaround it by hard-coding can_share_own_pool to 1 in gst1-plugins-good/sys/v4l2/gstv4l2object.c

@chrisiberg
Copy link
Author

Thnx I will try this.

the v4l2src is the HDMI-IN of rk3588 "hdmirx"

@chrisiberg
Copy link
Author 

DISPLAY=:0 gst-launch-1.0 v4l2src device=/dev/video0 ! glupload ! glcolorconvert ! glcolorscale ! 'video/x-raw(memory:GLMemory)', format=RGBA, width=3840, height=2160 ! gldownload ! glimagesink > /tmp/gst
MESA-LOADER: failed to open rockchip: /usr/lib/dri/rockchip_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri)
failed to load driver: rockchip
MESA-LOADER: failed to open rockchip: /usr/lib/dri/rockchip_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri)
failed to load driver: rockchip
0:00:00.237009918 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:545:gst_v4l2_buffer_pool_set_config:<v4l2src0:pool0:src> config GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\,\ format\=\(string\)BGR\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)120/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:1:7:1", size=(uint)6220800, min-buffers=(uint)0, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL;
0:00:00.237044917 32549   0x7f5c000b70 INFO          v4l2bufferpool gstv4l2bufferpool.c:593:gst_v4l2_buffer_pool_set_config:<v4l2src0:pool0:src> increasing minimum buffers to 2
0:00:00.237056291 32549   0x7f5c000b70 INFO          v4l2bufferpool gstv4l2bufferpool.c:606:gst_v4l2_buffer_pool_set_config:<v4l2src0:pool0:src> reducing maximum buffers to 64
0:00:00.237101789 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:545:gst_v4l2_buffer_pool_set_config:<v4l2src0:pool0:src> config GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\,\ format\=\(string\)BGR\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)120/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:1:7:1", size=(uint)6220800, min-buffers=(uint)3, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL, options=(string)< GstBufferPoolOptionVideoMeta >;
0:00:00.237122205 32549   0x7f5c000b70 INFO          v4l2bufferpool gstv4l2bufferpool.c:606:gst_v4l2_buffer_pool_set_config:<v4l2src0:pool0:src> reducing maximum buffers to 64
0:00:00.237149621 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:545:gst_v4l2_buffer_pool_set_config:<v4l2src0:pool0:src> config GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\,\ format\=\(string\)BGR\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ framerate\=\(fraction\)120/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:1:7:1", size=(uint)6220800, min-buffers=(uint)3, max-buffers=(uint)64, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL, options=(string)< GstBufferPoolOptionVideoMeta >;
0:00:00.237199202 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:791:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> activating pool
0:00:00.237210285 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:838:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> requesting 3 MMAP buffers
0:00:00.240749225 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f540091b0 (queued = no)
0:00:00.240784224 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f54009500 (queued = no)
0:00:00.240804056 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f54008620 (queued = no)
0:00:00.240848680 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:709:gst_v4l2_buffer_pool_streamon:<v4l2src0:pool0:src> Started streaming
0:00:00.240857721 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1103:gst_v4l2_buffer_pool_flush_stop:<v4l2src0:pool0:src> stop flushing
0:00:00.240878137 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1513:gst_v4l2_buffer_pool_acquire_buffer:<v4l2src0:pool0:src> acquire
0:00:00.359070317 32549   0x7f5c000b70 WARN          v4l2bufferpool gstv4l2bufferpool.c:1397:gst_v4l2_buffer_pool_dqbuf:<v4l2src0:pool0:src> Driver should never set v4l2_buffer.field to ANY
0:00:00.359116107 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1973:gst_v4l2_buffer_pool_process:<v4l2src0:pool0:src> process buffer 0x7f540091b0
0:00:01.026645149 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f540091b0 (queued = no)
0:00:04.710216143 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1513:gst_v4l2_buffer_pool_acquire_buffer:<v4l2src0:pool0:src> acquire
0:00:04.710265141 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1973:gst_v4l2_buffer_pool_process:<v4l2src0:pool0:src> process buffer 0x7f54009500
0:00:09.336884524 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f54009500 (queued = no)
0:00:10.971165072 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1513:gst_v4l2_buffer_pool_acquire_buffer:<v4l2src0:pool0:src> acquire
0:00:10.971284650 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1973:gst_v4l2_buffer_pool_process:<v4l2src0:pool0:src> process buffer 0x7f54008620
0:00:10.971375647 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f54008620 (queued = no)
0:00:10.971431644 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1513:gst_v4l2_buffer_pool_acquire_buffer:<v4l2src0:pool0:src> acquire
0:00:10.971485892 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1973:gst_v4l2_buffer_pool_process:<v4l2src0:pool0:src> process buffer 0x7f540091b0
0:00:11.548104578 32549   0x7f5c000b70 DEBUG         v4l2bufferpool gstv4l2bufferpool.c:1610:gst_v4l2_buffer_pool_complete_release_buffer:<v4l2src0:pool0:src> complete release buffer 0x7f540091b0 (queued = no)

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 2, 2024

there're some hack patches for glupload to import dmabuf directly without memcpy():
https://github.com/JeffyCN/meta-rockchip/tree/master/recipes-multimedia/gstreamer/gstreamer1.0-plugins-base_1.22.9

make sure that they are applied.

export GST_DEBUG=glupload:5
and check log: attempting upload with uploader DirectDmabufExternal

@chrisiberg
Copy link
Author

is it possible, that the symbolic links from 1.22.9 to 1.22.11 don't work?

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 2, 2024

normally it should work, since you said the 1.20.7 also failed.

maybe you can check the build/tmp/work//gstgood/*/temp/log.do_patch about which patches been applied

@chrisiberg
Copy link
Author

I freshly cloned meta-rockchip, because I already did some mess.

I had to remove the patch
recipes-kernel/linux/linux-rockchip_6.1/0004-media-rockchip-vicap-fixes-error-of-gcc-compile.patch
because it seems this is already done here.
https://github.com/friendlyarm/kernel-rockchip/blob/nanopi6-v6.1.y/drivers/media/platform/rockchip/cif/capture.c

gst patches seem all to be done.
But the problem is not solved.

GST_DEBUG=glupload:5

Setting pipeline to PAUSED ...
MESA-LOADER: failed to open rockchip: /usr/lib/dri/rockchip_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri)
failed to load driver: rockchip
MESA-LOADER: failed to open rockchip: /usr/lib/dri/rockchip_dri.so: cannot open shared object file: No such file or directory (search paths /usr/lib/dri, suffix _dri)
failed to load driver: rockchip
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.322878966 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload1> attempting upload with uploader GLMemory
0:00:00.322915117 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload1> attempting upload with uploader DirectDmabufExternal
0:00:00.322928237 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload1> attempting upload with uploader DirectDmabuf
0:00:00.322938149 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload1> attempting upload with uploader Dmabuf
0:00:00.322947770 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload1> attempting upload with uploader UploadMeta
0:00:00.322961764 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload1> attempting upload with uploader Raw Data
0:00:02.516616933 15931   0x7f80000b70 DEBUG               glupload gstglupload.c:2551:_upload_find_method:<glupload0> attempting upload with uploader GLMemory
Redistribute latency...

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 2, 2024

you can upload your rootfs.img, maybe i can try to check it.

@chrisiberg
Copy link
Author

chrisiberg commented Jul 2, 2024 •
edited
Loading

I sent a fast shot, I hope you can use it.
https://drive.google.com/file/d/14DGc65808G1IiVn1UdAjgSbutAawavkE/view?usp=drive_link

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

try:
export GST_V4L2_MIN_BUFS=24

could be buffer not enough

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024 •
edited
Loading

another try is to skip sync in kernel:

+++ b/drivers/media/common/videobuf2/videobuf2-cma-sg.c
@@ -242,7 +242,7 @@ static void vb2_cma_sg_prepare(void *buf_priv)
        struct vb2_cma_sg_buf *buf = buf_priv;
        struct sg_table *sgt = buf->dma_sgt;

-       if (buf->vb->skip_cache_sync_on_prepare)
+       if (1 || buf->vb->skip_cache_sync_on_prepare)
                return;

the proper way is to set V4L2_BUF_FLAG_NO_CACHE_CLEAN in the v4l2src.

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

Also try to set FlipFB to none in:
/usr/share/X11/xorg.conf.d/20-modesetting.conf

which would affect xserver render performance.

@chrisiberg
Copy link
Author

Nothing helped.
I missed to try "can_share_own_pool to 1 in gst1-plugins-good/sys/v4l2/gstv4l2object.c"
will try now.

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

so the first a few frames in glimagesink is slow(due to x11 window initialization), the gst will try to sync them and causing those delays.

you can try to add sync=false in your pipeline's glimagesink to avoid syncing (with system clock or audio)

@chrisiberg
Copy link
Author

chrisiberg commented Jul 3, 2024 •
edited
Loading

gst-launch-1.0 v4l2src device=/dev/video0 ! glupload ! glcolorconvert ! glcolorscale ! 'video/x-raw(memory:GLMemory)', format=RGBA, width=3840, height=2160 ! gldownload ! glimagesink sync=false

no change, the framrate is about 1 in 5sec

I s it possible there is a problem with hdmirx?

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

could be, you can try to test it with video:
gst-launch-1.0 uridecodebin uri=file:///

@chrisiberg
Copy link
Author

ok, not hdmirx, same problem with a video file.

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

it works well on my side with camera and video.

i have these environments:
export DISPLAY=:0
export GST_GL_API="gles2"
export GST_GL_PLATFORM="egl"
export GST_V4L2_MIN_BUFS=16
unset GST_V4L2_USE_LIBV4L2

and with above hacks.

@chrisiberg
Copy link
Author

Thnx a lot, this works!
But I have to work out which on of these environments did it.

@chrisiberg
Copy link
Author

I have to set both
export GST_GL_API="gles2"
export GST_GL_PLATFORM="egl"

@chrisiberg
Copy link
Author

are they usual set in a script, did I mess up something?

@chrisiberg
Copy link
Author

I will rebuild all without the patches, and try it again.

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

the Mali DDK(GPU library) doesn't support OpenGL, it only support EGL and OpenGLESv2, so if the gst has OpenGL enabled, it might use mesa3d(software compositor in this case).

@JeffyCN
Copy link
Owner

JeffyCN commented Jul 3, 2024

You can set these in /etc/profile.d/gst.sh, or set it in the gst-plugin-base's configure arguments:

EXTRA_OEMESON += "
-Ddoc=disabled
${@get_opengl_cmdline_list('gl_api', d.getVar('OPENGL_APIS'), d)}
${@get_opengl_cmdline_list('gl_platform', d.getVar('OPENGL_PLATFORMS'), d)}
${@get_opengl_cmdline_list('gl_winsys', d.getVar('OPENGL_WINSYS'), d)}
"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JeffyCN @chrisiberg