Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exclude libOpenGL.so.0* #152

Closed
okaerin opened this issue Oct 7, 2020 · 24 comments
Closed

exclude libOpenGL.so.0* #152

okaerin opened this issue Oct 7, 2020 · 24 comments
Labels
bug Something isn't working confirmed

Comments

@okaerin
Copy link

okaerin commented Oct 7, 2020

I have a Qt5 project where libOpenGL.so.0.0.0 gets deployed into the appimage by linuxdeploy. When starting the appimage afterwards I get this warning from Qt:
Unrecognized OpenGL version
If I preload my machines libOpenGL.so.0.0.0 it works fine. Would it be possible to exclude this shared library ?

@TheAssassin
Copy link
Member

Many OpenGL libraries are excluded already. See https://github.com/AppImage/pkg2appimage/blob/master/excludelist#L38.
I guess it would be fine to exclude this lib as well.

Can you please check how libOpenGL is pulled in? You can use lddtree for that. I could imagine that none of your code uses it directly, but through one of the already-excluded libraries. In that case, this might be a duplicate of #83.

@okaerin
Copy link
Author

okaerin commented Oct 9, 2020

I noticed that a few OpenGL libraries are already there and thought it may be related to this issue as well #89 .
Somehow it gets pulled in by my toplevel application

application => ./application (interpreter => /lib64/ld-linux-x86-64.so.2)
    libhip_seg.so => ./../lib/libhip_seg.so
        libQt5WebSockets.so.5 => ./../lib/libQt5WebSockets.so.5
    libQt5Quick.so.5 => ./../lib/libQt5Quick.so.5
        libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1
            libGLdispatch.so.0 => /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0
    libQt5Qml.so.5 => ./../lib/libQt5Qml.so.5
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
    libQt5Network.so.5 => ./../lib/libQt5Network.so.5
    libQt5PrintSupport.so.5 => ./../lib/libQt5PrintSupport.so.5
    libQt5Widgets.so.5 => ./../lib/libQt5Widgets.so.5
    libQt5Gui.so.5 => ./../lib/libQt5Gui.so.5
    libQt5Core.so.5 => ./../lib/libQt5Core.so.5
        libicui18n.so.56 => ./../lib/libicui18n.so.56
        libicuuc.so.56 => ./../lib/libicuuc.so.56
        libicudata.so.56 => ./../lib/libicudata.so.56
        libgthread-2.0.so.0 => ./../lib/libgthread-2.0.so.0
        libglib-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
            libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3
        ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
    libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6
        libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1
            libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6
            libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6
                libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0
    libXt.so.6 => ./../lib/libXt.so.6
        libSM.so.6 => /usr/lib/x86_64-linux-gnu/libSM.so.6
            libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1
        libICE.so.6 => /usr/lib/x86_64-linux-gnu/libICE.so.6
    libOpenGL.so.0 => ./../lib/libOpenGL.so.0
    libGLX.so.0 => /usr/lib/x86_64-linux-gnu/libGLX.so.0
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6

TheAssassin added a commit to AppImageCommunity/pkg2appimage that referenced this issue Oct 9, 2020
@TheAssassin
Copy link
Member

An investigation conducted on IRC shows that it's Qt itself linking to libOpenGL. This Qt was installed via install-qt.sh, provided by the upstream Qt company. The Ubuntu distro packages (and beineri's PPA packages) link to libGL (instead?), leading to this issue. I've updated the excludelist (see AppImageCommunity/pkg2appimage@c8e9eba), and will rebuild linuxdeploy and the Qt plugin.

@TheAssassin
Copy link
Member

Excluding the library worked. Closing as resolved.

@TheAssassin TheAssassin added bug Something isn't working confirmed labels Oct 9, 2020
@hcorion
Copy link

hcorion commented May 1, 2021

So it seems that distros running newer Ubuntu versions (and likely Ubuntu itself), like Pop! OS no longer bundle libOpenGL.so.0 as part of their default installation. We ran into issues with RPCS3's AppImage running on these distros. We fixed it by bundling libOpenGL.so.0 manually, so it may be time to revisit this issue?
Relevant issue: RPCS3/rpcs3#10226

@okaerin
Copy link
Author

okaerin commented May 2, 2021

@hcorion

The points is that libOpenGL.so.0 is rather a "system specific" library (different versions required with different OS versions/distributions) and is therefore not really portable. I would strongly discourage to bundle libObenGL.so.0 with the AppImage. In my case I was building for Ubuntu 16.04 (while it was still bundling the library) and I could not execute the AppImage on an Ubuntu 20.04. What you are probably missing is the package libopengl-dev within your CI machine. You need to link to it, but it shouldn't get bundled with

@hcorion
Copy link

hcorion commented May 2, 2021

What?
We're building on Ubuntu 18.04, as per the AppImage standard of building on the lowest supported LTS Ubuntu version, which doesn't contain libopengl-dev (introduced in 20.04), and instead has libopengl0, which we already have installed.
I don't understand how that would solve the problem though? The issue is that libOpenGL.so.0 does not exist by default on Ubuntu 20.04+, defeating the point of AppImages as a plug and play solution.

@TheAssassin
Copy link
Member

TheAssassin commented May 2, 2021

Which of your libraries pulls in libOpenGL? Normally, you are talking to libGL, libEGL, libDRM and other stuff. I've never heard of software linking to some libOpenGL. Please call lddtree on your binariy and share the result.

Hint: I suspect this to be a general dependency problem in your build chain.

@hcorion
Copy link

hcorion commented May 2, 2021

RPCS3 is a 3D application, so we use OpenGL directly, it's linked at the top level.

Output of lddtree on Ubuntu 18.04

rpcs3 => squashfs-root/usr/bin/rpcs3 (interpreter => /lib64/ld-linux-x86-64.so.2)      
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1                                     
    libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6                               
        libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1                           
            libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6                       
            libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6                   
                libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0                       
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2                                     
    libevdev.so.2 => squashfs-root/usr/bin/../lib/libevdev.so.2                        
    libasound.so.2 => /usr/lib/x86_64-linux-gnu/libasound.so.2                         
    libpulse-simple.so.0 => squashfs-root/usr/bin/../lib/libpulse-simple.so.0          
        libpulsecommon-11.1.so => squashfs-root/usr/bin/../lib/libpulsecommon-11.1.so  
            libsystemd.so.0 => squashfs-root/usr/bin/../lib/libsystemd.so.0            
                liblzma.so.5 => squashfs-root/usr/bin/../lib/liblzma.so.5              
                liblz4.so.1 => squashfs-root/usr/bin/../lib/liblz4.so.1                
                libgcrypt.so.20 => squashfs-root/usr/bin/../lib/libgcrypt.so.20        
                    libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0       
            libwrap.so.0 => squashfs-root/usr/bin/../lib/libwrap.so.0                  
                libnsl.so.1 => squashfs-root/usr/bin/../lib/libnsl.so.1                
            libsndfile.so.1 => squashfs-root/usr/bin/../lib/libsndfile.so.1            
                libFLAC.so.8 => squashfs-root/usr/bin/../lib/libFLAC.so.8              
                libogg.so.0 => squashfs-root/usr/bin/../lib/libogg.so.0                
                libvorbis.so.0 => squashfs-root/usr/bin/../lib/libvorbis.so.0          
                libvorbisenc.so.2 => squashfs-root/usr/bin/../lib/libvorbisenc.so.2    
            libasyncns.so.0 => squashfs-root/usr/bin/../lib/libasyncns.so.0            
                libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2                 
            libdbus-1.so.3 => squashfs-root/usr/bin/../lib/libdbus-1.so.3              
            libapparmor.so.1 => squashfs-root/usr/bin/../lib/libapparmor.so.1          
    libpulse.so.0 => squashfs-root/usr/bin/../lib/libpulse.so.0                        
    libSDL2-2.0.so.0 => squashfs-root/usr/bin/../lib/libSDL2-2.0.so.0                  
    libopenal.so.1 => squashfs-root/usr/bin/../lib/libopenal.so.1                      
        libsndio.so.6.1 => squashfs-root/usr/bin/../lib/libsndio.so.6.1                
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1                                       
    libtinfo.so.5 => squashfs-root/usr/bin/../lib/libtinfo.so.5                        
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0                           
    libOpenGL.so.0 => squashfs-root/usr/bin/../lib/libOpenGL.so.0                      
        libGLdispatch.so.0 => /usr/lib/x86_64-linux-gnu/libGLdispatch.so.0             
    libGLX.so.0 => /usr/lib/x86_64-linux-gnu/libGLX.so.0                               
    libvulkan.so.1 => squashfs-root/usr/bin/../lib/libvulkan.so.1                      
    libGLEW.so.2.2 => squashfs-root/usr/bin/../lib/libGLEW.so.2.2                      
        libGL.so.1 => /usr/lib/x86_64-linux-gnu/libGL.so.1                             
    libQt5Widgets.so.5 => squashfs-root/usr/bin/../lib/libQt5Widgets.so.5              
    libQt5Gui.so.5 => squashfs-root/usr/bin/../lib/libQt5Gui.so.5                      
        libpng16.so.16 => squashfs-root/usr/bin/../lib/libpng16.so.16                  
        libharfbuzz.so.0 => /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0                 
            libglib-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0             
                libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3                     
            libfreetype.so.6 => /usr/lib/x86_64-linux-gnu/libfreetype.so.6             
            libgraphite2.so.3 => /usr/lib/x86_64-linux-gnu/libgraphite2.so.3           
    libQt5DBus.so.5 => squashfs-root/usr/bin/../lib/libQt5DBus.so.5                    
    libQt5Concurrent.so.5 => squashfs-root/usr/bin/../lib/libQt5Concurrent.so.5        
    libQt5Core.so.5 => squashfs-root/usr/bin/../lib/libQt5Core.so.5                    
        libicui18n.so.60 => squashfs-root/usr/bin/../lib/libicui18n.so.60              
        libicuuc.so.60 => squashfs-root/usr/bin/../lib/libicuuc.so.60                  
            libicudata.so.60 => squashfs-root/usr/bin/../lib/libicudata.so.60          
    libudev.so.1 => squashfs-root/usr/bin/../lib/libudev.so.1                          
    libcurl.so.4 => squashfs-root/usr/bin/../lib/libcurl.so.4                          
        libnghttp2.so.14 => squashfs-root/usr/bin/../lib/libnghttp2.so.14              
        libidn2.so.0 => squashfs-root/usr/bin/../lib/libidn2.so.0                      
            libunistring.so.2 => squashfs-root/usr/bin/../lib/libunistring.so.2        
        librtmp.so.1 => squashfs-root/usr/bin/../lib/librtmp.so.1                      
            libgnutls.so.30 => squashfs-root/usr/bin/../lib/libgnutls.so.30            
                libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0           
                    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6               
                libtasn1.so.6 => squashfs-root/usr/bin/../lib/libtasn1.so.6            
            libhogweed.so.4 => squashfs-root/usr/bin/../lib/libhogweed.so.4            
            libnettle.so.6 => squashfs-root/usr/bin/../lib/libnettle.so.6              
            libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10                     
        libpsl.so.5 => squashfs-root/usr/bin/../lib/libpsl.so.5                        
        libssl.so.1.1 => squashfs-root/usr/bin/../lib/libssl.so.1.1                    
        libcrypto.so.1.1 => squashfs-root/usr/bin/../lib/libcrypto.so.1.1              
        libgssapi_krb5.so.2 => squashfs-root/usr/bin/../lib/libgssapi_krb5.so.2        
            libkrb5.so.3 => squashfs-root/usr/bin/../lib/libkrb5.so.3                  
                libkeyutils.so.1 => squashfs-root/usr/bin/../lib/libkeyutils.so.1      
            libk5crypto.so.3 => squashfs-root/usr/bin/../lib/libk5crypto.so.3          
            libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2                   
            libkrb5support.so.0 => squashfs-root/usr/bin/../lib/libkrb5support.so.0    
        libldap_r-2.4.so.2 => squashfs-root/usr/bin/../lib/libldap_r-2.4.so.2          
            libsasl2.so.2 => squashfs-root/usr/bin/../lib/libsasl2.so.2                
            libgssapi.so.3 => squashfs-root/usr/bin/../lib/libgssapi.so.3              
                libheimntlm.so.0 => squashfs-root/usr/bin/../lib/libheimntlm.so.0      
                    libwind.so.0 => squashfs-root/usr/bin/../lib/libwind.so.0          
                libkrb5.so.26 => squashfs-root/usr/bin/../lib/libkrb5.so.26            
                    libheimbase.so.1 => squashfs-root/usr/bin/../lib/libheimbase.so.1  
                    libhx509.so.5 => squashfs-root/usr/bin/../lib/libhx509.so.5        
                    libsqlite3.so.0 => squashfs-root/usr/bin/../lib/libsqlite3.so.0    
                    libcrypt.so.1 => squashfs-root/usr/bin/../lib/libcrypt.so.1        
                libasn1.so.8 => squashfs-root/usr/bin/../lib/libasn1.so.8              
                libhcrypto.so.4 => squashfs-root/usr/bin/../lib/libhcrypto.so.4        
                libroken.so.18 => squashfs-root/usr/bin/../lib/libroken.so.18          
        liblber-2.4.so.2 => squashfs-root/usr/bin/../lib/liblber-2.4.so.2              
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6                                       
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6                         
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1                               
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6                                       
    ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2                                
    

@hcorion
Copy link

hcorion commented May 2, 2021

Hmmm, looking into it further, we didn't used to link directly libOpenGL when we built on 16.04 🤔. Not sure what has changed on 18.04

@TheAssassin
Copy link
Member

Well, that's the question I was trying to find an answer for. Your build system (or its dependency lookup stuff) clearly pulls it in as a direct dependency. I am not sure what makes it link to some libOpenGL. As said, usually, apps link to libGL.

@okaerin
Copy link
Author

okaerin commented May 2, 2021

Browsing through the build system I found in your 3rd party CmakeLists that you try to find opengl and you want to prefer glvnd (if available, atleast what the cmake docs say) and else use lib opengl. so maybe just libglvnd is missing after 18.04?

@TheAssassin
Copy link
Member

I checked all the libraries in my workstation's /usr/lib/x86_64-linux-gnu, and none of these libraries appears to link to some libOpenGL...

@TheAssassin
Copy link
Member

According to some issues on the CMake issue tracker, some forums etc., it seems like you really should try to make your build system link to GLVND again. I suppose otherwise you'd want to link against libGL. I haven't had a look at your code, this assumes you do the dependency lookup in your own code. The original issue discussed here was the kind of broken custom Qt build system vs. distro-provided libraries. libOpenGL seems to be used by hardly any library (I quickly ran apt-cache rdepends libopengl0 to check that vs libgl1).

I can't imagine your problem would be solved properly if you bundled libOpenGL. You were just checking on Ubuntu, but try other distros as well. There is a reason we exclude close-to-hardware libraries.

@hcorion
Copy link

hcorion commented May 2, 2021

We should be linking against libglvnd, and we have libglvnd0 installed on the CI build configuration.
Our CMakeLists.txt looks like this, and hasn't changed since our upgrade to 16.04

# Prefer GLVND for OpenGL rather than legacy
set(OpenGL_GL_PREFERENCE GLVND)
find_package(OpenGL REQUIRED)

add_library(3rdparty_opengl INTERFACE)
target_include_directories(3rdparty_opengl INTERFACE GL)

if (WIN32)
// Irrelevant
else()
	target_link_libraries(3rdparty_opengl INTERFACE ${OPENGL_LIBRARIES})

	target_compile_definitions(3rdparty_opengl
		INTERFACE
			-DGL_GLEXT_PROTOTYPES
			-DGLX_GLXEXT_PROTOTYPES)
endif()

@TheAssassin
Copy link
Member

But apparently you link libOpenGL directly, which is most likely hidden in OPENGL_LIBRARIES. Check your CMakeCache.txt for the actual value. What version of CMake are you using? There have been changes, look into the CMake issues for more information.

@hcorion
Copy link

hcorion commented May 2, 2021

We're using CMake 3.16.9, do you think you could direct me towards some of the issues you mentioned so I can investigate what version they were fixed in?

@hcorion
Copy link

hcorion commented May 2, 2021

Looks like libglvnd-dev was the package that we need, testing now

@TheAssassin
Copy link
Member

Mostly https://gitlab.kitware.com/cmake/cmake/-/issues/17437. Since CMake 3.10, I think it's best practice to just use the imported targets anyway. Linking to the entity of OPENGL_LIBRARIES is a deprecated approach.

@hcorion
Copy link

hcorion commented May 2, 2021

Hmmm, so it seems like the GLVND approach might be linking against libOpenGL.so.0, doing more testing...

@TheAssassin
Copy link
Member

TheAssassin commented May 3, 2021

# value of OPENGL_LIBRARIES in RPCS3 CMake script
/usr/lib/x86_64-linux-gnu/libOpenGL.so;/usr/lib/x86_64-linux-gnu/libGLX.so;/usr/lib/x86_64-linux-gnu/libGLU.so

So indeed, it's the CMake OpenGL discovery that leads to linking this library directly. Note this is on a random Ubuntu 20.04 with CMake 3.20.1.

Looking at https://cmake.org/cmake/help/latest/module/FindOpenGL.html reveals that it's most likely caused by using GLVND, there's multiple hints on there. set(OpenGL_GL_PREFERENCE LEGACY) makes it prefer libGL instead, which apparently is a "legacy" library, according to the CMake docs. Can you perhaps use that legacy library instead, even if just for AppImage builds?

Edit: We won't be able to bundle libOpenGL, so I'm looking for a workaround for your issue. Note it's still recommended to use the imported targets rather than ${OPENGL_LIBRARIES}.

@hcorion
Copy link

hcorion commented May 3, 2021

legacy seems to have worked 🤔. If we use the imported targets do you think it'll work with GLVND?

@TheAssassin
Copy link
Member

I think if you want to go for this legacy option, you need to use OpenGL::GL. I'm not an expert with these libraries, just reading the docs. I think as long as you can use legacy, go for it. It'll most likely fix your issue.

I think we have to take a deeper look overall into this GLVND stuff. This is actually the wrong place to discuss decisions regarding the excludelist, it would be more appropriate in https://github.com/AppImage/pkg2appimage.

Reading the docs, I am relatively sure libOpenGL is hardware-independent. If GLVND is used, libGL etc. are also just wrappers around libOpenGL. The main issue seems to be that if you'd bundle libOpenGL, you have to make sure the target system is providing GLVND as well. This has not been the case in the past with some distros, and it seems like some dists like Pop also no longer make sure it's available. I'm sure libGL is available virtually everywhere, though.

I think the recommendation for making AppImages has to be "use legacy". It should work with both GLVND and the old system.

@TheAssassin
Copy link
Member

For the record, I also could build RPCS3 just fine with the legacy option. I guess you can safely go for it. But I'd highly appreciate if you created an issue over at pkg2appimage to discuss the issue more thoroughly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working confirmed
Projects
None yet
Development

No branches or pull requests

3 participants