Mediapipe GPU support for wayland-egl

[pkoppise]

Dear Team,

I am able to run hand tracking example (CPU) on the RPI4, I also want to run the same example on GPU whose configuration is as follows:

OS: webOS
Graphics card: Broadcom vc4
Display: wayland

The driver versions we are using to support vc4 are
Mesa21.1.5
libdrm107
libGLESv2
OpenGL ES 3.1

After looking into gpu code looks like mediapipe supports egl applications, could you please provide any inputs on how to proceed with wayland-egl with mediapipe

FYI: I have started modifying gpu code of mediapipe on my own, but it's crashing

I20211228 17:05:48.519873 1544 gl_context_egl.cc:198] Successfully initialized EGL. Major : 1 Minor: 4
I20211228 17:05:48.519950 1544 gl_context_egl.cc:95] number of configs found 32
I20211228 17:05:48.519997 1544 gl_context_egl.cc:114] Total: 32 Matching attribs: 6
I20211228 17:05:48.520018 1544 gl_context_egl.cc:133] Config ID 10
I20211228 17:05:48.520038 1544 gl_context_egl.cc:136] Depth Size 16
I20211228 17:05:48.520057 1544 gl_context_egl.cc:139] Red Size 8
I20211228 17:05:48.520076 1544 gl_context_egl.cc:142] Green Size 8
I20211228 17:05:48.520093 1544 gl_context_egl.cc:145] Blue Size 8
I20211228 17:05:48.520112 1544 gl_context_egl.cc:148] Alpha Size 8
I20211228 17:05:48.520131 1544 gl_context_egl.cc:151] Renderable Type 77
I20211228 17:05:48.520150 1544 gl_context_egl.cc:154] Surface Type 4
I20211228 17:05:48.520167 1544 gl_context_egl.cc:165] Creating GL Context
I20211228 17:05:48.523841 1544 gl_context_egl.cc:176] Done GL Context

Thread 1 "qml-runner" received signal SIGSEGV, Segmentation fault.
0x0000007ff50ff694 in wl_proxy_marshal_constructor () from /usr/lib/libwayland-client.so.0
(gdb) bt
#0 0x0000007ff50ff694 in wl_proxy_marshal_constructor () from /usr/lib/libwayland-client.so.0
1 0x0000007fea5cb960 in mediapipe::GlContext::CreateContext(void*) () from /usr/lib/libmediapipegpu.so.0
2 0x0000007fea5cbf1c in mediapipe::GlContext::Create(void*, bool) () from /usr/lib/libmediapipegpu.so.0
3 0x0000007fea59933c in mediapipe::GpuResources::Create(void*) () from /usr/lib/libmediapipegpu.so.0
4 0x0000007fea59982c in mediapipe::GpuResources::Create() () from /usr/lib/libmediapipegpu.so.0

[sgowroji]

Hi @pkoppise, Do you see "Thread 1 "qml-runner" received signal SIGSEGV, Segmentation fault." error while running the above mediapipe build on GPU ?

[pkoppise]

yeah, segmentation fault is happening, looks like the code i have written / modified to intialize wayland-egl is causing this issue

[pkoppise]

Looks like , the way to initialize different egl is different for different platforms.

Currently to my knowledge mediapipe is allocating gpu resources for Linux platfroms (gbm ) which supports OpenGL ES3.1?

Not sure whether same gpu code works for other platforms (wayland) . Could you please help?

[sgowroji]

For GPU working and support you can refer this. https://google.github.io/mediapipe/framework_concepts/gpu.html. We have not tried any with the above GPU device may be you can test and let us know the details.

[google-ml-butler]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No