obs-glcapture Fails to Capture

[ibrokemypie]

When trying to use obs-glcapture to capture anything (including glxgears) I just get this output:

[obs-vkcapture] Init 0.7
297 shaders loaded in 0.3716s
[obs-vkcapture] Texture GL_RGBA 1920x1080
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed
[obs-vkcapture] Texture GL_RGBA 1920x1080
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed
[obs-vkcapture] Texture GL_RGBA 1920x1080
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed
[obs-vkcapture] Texture GL_RGBA 1920x1080
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed
[obs-vkcapture] Texture GL_RGBA 1920x1080
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed

This is on an amd vega card with the latest mesa and amdgpu drivers

[nowrep]

Do you have DRI3 enabled?

xdpyinfo | grep DRI3

[ibrokemypie]

yep

xdpyinfo | grep DRI3
    DRI3

[nowrep]

diff --git a/src/glinject.c b/src/glinject.c
index 7bdc8f8..abedc57 100644
--- a/src/glinject.c
+++ b/src/glinject.c
@@ -28,6 +28,8 @@ with this program. If not, see <https://www.gnu.org/licenses/>
 #include <unistd.h>
 #include <stdlib.h>
 
+#include <xcb/xcb.h>
+
 static bool gl_seen = false;
 static bool gl_loaded = false;
 static struct egl_funcs egl_f;
@@ -313,8 +315,10 @@ static bool gl_shtex_init()
         glx_f.BindTexImageEXT(data.display, data.glxpixmap, P_GLX_FRONT_LEFT_EXT, NULL);
 
         void *cookie = x11_f.xcb_dri3_buffer_from_pixmap(data.xcb_con, data.xpixmap);
-        P_xcb_dri3_buffer_from_pixmap_reply_t *reply = x11_f.xcb_dri3_buffer_from_pixmap_reply(data.xcb_con, cookie, NULL);
+        xcb_generic_error_t *err;
+        P_xcb_dri3_buffer_from_pixmap_reply_t *reply = x11_f.xcb_dri3_buffer_from_pixmap_reply(data.xcb_con, cookie, &err);
         if (!reply) {
+            hlog("error: %d", err->error_code);
             hlog("Failed to get buffer from pixmap");
             return false;
         }

Can you post output with this patch applied?

[ibrokemypie]

The output is

[obs-vkcapture] Init 0.7
[obs-vkcapture] Texture GL_RGBA 920x490
[obs-vkcapture] error: 4
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed
[obs-vkcapture] Texture GL_RGBA 920x490
[obs-vkcapture] error: 4
[obs-vkcapture] Failed to get buffer from pixmap
[obs-vkcapture] shtex init failed
[obs-vkcapture] gl_init failed
[obs-vkcapture] Texture GL_RGBA 920x490
[obs-vkcapture] error: 4

[nowrep]

I'm not really sure how to debug this, but just to make sure: Does it work with EGL (obs-glcapture eglgears_x11) and Vulkan (obs-vkcapture vkcube)?

[nowrep]

Ok I actually got hit by this, when running on XWayland. Any chance it was on XWayland in your case too?

[nowrep]

Not supported on XWayland:
https://gitlab.freedesktop.org/xorg/xserver/-/blob/xwayland-21.1/hw/xwayland/xwayland-glamor-gbm.c#L553