Doesn't work with Mangohud

[Hubro]

obs-glcapture doesn't seem to work when used at the same time as Mangohud. Instead, the outermost invocation gets precedence. For example, when running a OpenGL Steam game like:

obs-glcapture mangohud %command%

Game capture works, but Mangohud doesn't work.

If I use:

mangohud obs-glcapture %command%

Now Mangohud works, but game captures doesn't. (I just get a black screen.)

Is it possible to make them work at the same time?

[nowrep]

I don't think so, both use dlsym. Similarly I think it can't work with Steam overlay either.

[Hubro]

I don't think so, both use dlsym. Similarly I think it can't work with Steam overlay either.

Actually Steam overlay does work while I'm recording. It even gets included in the recording.

Screenshot from OBS:

[nowrep]

Hmm, then maybe Steam is hooking the functions without dlsym.
I think it's not possible to chain hooks with dlsym, but I am not absolutely sure. Does mangohud + libstrangle work together?

Also one thing to note, mangohud by default isn't using dlsym - you have to use mangohud --dlsym ....

[nowrep]

You can try this

diff --git a/src/glinject.c b/src/glinject.c
index e4b22a2..c9fb3c8 100644
--- a/src/glinject.c
+++ b/src/glinject.c
@@ -16,6 +16,8 @@ You should have received a copy of the GNU General Public License along
 with this program. If not, see <https://www.gnu.org/licenses/>
 */
 
+#define _GNU_SOURCE
+
 #include "glinject.h"
 #include "capture.h"
 #include "utils.h"
@@ -59,7 +61,10 @@ struct gl_data {
 static struct gl_data data;
 
 #define GETADDR(s, p, func) \
-    p.func = (typeof(p.func))real_dlsym(handle, #s #func); \
+    p.func = (typeof(p.func))real_dlsym(RTLD_NEXT, #s #func); \
+    if (!p.func) { \
+        p.func = (typeof(p.func))real_dlsym(handle, #s #func); \
+    } \
     if (!p.func) { \
         hlog("Failed to resolve " #s #func); \
         return false; \

Problem with this is that it makes mangohud + alacritty EGL crash. It will crash when trying to call GL functions obtained from mangohud's eglGetProcAddress.
And with libstrangle + glxgears glXGetProcAddress will return functions from obs-glcapture which crashes it too.

Honestly it's a mess.

[Hubro]

Any idea how Steam overlay is working fine with all these other hooks? 🤔

[nowrep]

Not really, and of course there's no source code to look into.
It does use dlsym and seems to be calling the original functions resolved with dlsym(RTLD_NEXT, ...), but it also is linked to libGL. Maybe that's enough to make it work, hmmm.

[Hubro]

By the way, I tried mangohud --dlsym just to be sure, and it didn't make a difference.

[nowrep]

Didn't make a difference when I tried myself either.

With current master it should work now with mangohud, as long as obs-glcapture is before mangohud.

[Hubro]

Nice! I guess the perfect world scenario, where all OpenGL "overlay" applications work together no matter the order they are applied in, is impossible? 😄

Perhaps at some point we'll have a good translation layer to convert OpenGL calls to Vulkan on the fly, like DXVK does for DirectX 😄 (I wonder if Zink is in a working state for games yet?)

[nowrep]

Nice! I guess the perfect world scenario, where all OpenGL "overlay" applications work together no matter the order they are applied in, is impossible? smile

Yes, there is no standard way how to do it with OpenGL, as it is with Vulkan.

I've pushed another changes and it should now work fine with mangohud even for EGL. But I wonder what else will break.