New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raspbian integration (two libGLES side-by-side) #24

Closed
gohai opened this Issue Feb 15, 2016 · 42 comments

Comments

Projects
None yet
8 participants
@gohai

gohai commented Feb 15, 2016

@anholt: Please let me know if there is a more apt venue for reporting those! I was informally in contact with Matt from the Raspberry Pi foundation, told him about the we're having with the latest release, and he pointed me to your (amazing!) new feedback mechanism for the driver.

The recent Raspbian release ships with two libGLESv2.so implementations, which seems to break at least JOGL (2.3.2) as well as the GStreamer that comes with the distribution.

This seems to illustrate the problem:

ldconfig -p | grep GLES
libGLESv2.so.2 (libc6,hard-float) => /usr/lib/arm-linux-gnueabihf/libGLESv2.so.2
libGLESv2.so (libc6,hard-float) => /opt/vc/lib/libGLESv2.so

JOGL will ask the linker first for libGLESv2.so.2 (and would later fall back to libGLESv2.so) - so the Mesa one gets loaded, even though we're on BC IV.

Installing gstreamer1.0-tools and running gst-launch-1.0 playbin uri=file:///opt/vc/src/hello_pi/hello_video/test.h264 gives a SIGSEGV. Running sudo aptitude remove libgles1-mesa libgles2-mesa libgl1-mesa-dri magically solves this, suggesting a similar issue.

As developer, I want to be able to make my application run on the old, as well as the new driver - allowing the user to try both. JOGL is already capable of interfacing with both (as well as with other hardware) - the only thing that we'd need is a clear way of knowing which GLES{,2} implementation to use.

I'd love to just be able to use the linker (e.g. with some /etc/ld.so.conf reordering when enabling/disabling the new driver), to get the desired implementation. But it'd be good to know the plans in Raspbian for that, so that we can prepare.

@anholt

This comment has been minimized.

Show comment
Hide comment
@anholt

anholt Feb 16, 2016

Owner

If they're leaving both libGLESv2s in the link path, we're in trouble. Back when I was working with them in October, we'd talked a boot script moving the libraries such that only one could be found. If they did that, then in some glorious future you could autodetect by asking your EGL if it does X11 using EGL_EXT_platform_x11.txt (if it ever gets implemented). Until then you could hack it up by grepping the EGL vendor for "mesa" I guess.

Owner

anholt commented Feb 16, 2016

If they're leaving both libGLESv2s in the link path, we're in trouble. Back when I was working with them in October, we'd talked a boot script moving the libraries such that only one could be found. If they did that, then in some glorious future you could autodetect by asking your EGL if it does X11 using EGL_EXT_platform_x11.txt (if it ever gets implemented). Until then you could hack it up by grepping the EGL vendor for "mesa" I guess.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Feb 16, 2016

The former, both in the link path - raspi-config does nothing to this effect. You got a direct line to whoever's doing Raspbian?

Thanks for the pointers for detecting which on we got! (JOGL currently checks for the presence of /opt/vc/lib/bcm_host_init.so, and calls the platform-specific bcm_host_init. I'll make it so that we only do that if the vc4 module also isn't loaded..)

gohai commented Feb 16, 2016

The former, both in the link path - raspi-config does nothing to this effect. You got a direct line to whoever's doing Raspbian?

Thanks for the pointers for detecting which on we got! (JOGL currently checks for the presence of /opt/vc/lib/bcm_host_init.so, and calls the platform-specific bcm_host_init. I'll make it so that we only do that if the vc4 module also isn't loaded..)

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Mar 9, 2016

Eric, how many libraries are likely to have this problem of two copies installed? Which libraries does mesa / your driver add which are duplicates of existing VC libraries? I'm assuming this won't be the only one; whatever fix we come up with may as well deal with all of them.

We can't remove the old libraries; they are needed if the new driver is disabled, which it is by default. Is the right thing to do to adjust LD_LIBRARY_PATH on the system to change the order in which /opt/vc/lib and /usr/lib/arm-linux-gnueabihf appear depending on whether or not the new driver is enabled? If so, we can add that to raspi-config; is that going to be enough to fix this? Of course, that may also have unintended side effects elsewhere - I don't know what other applications are going to do based on a change in the order of the library path.

The alternative is to overwrite the libraries in /opt/vc/lib with the new ones, keeping backup copies; I have no idea what else that might break in the process.

Or would it be better to actually rename the new libraries completely so that both sets can coexist on the system? That strikes me as the cleanest approach and the least likely to cause longer-term problems - maybe with a relevant symlink being set up so that the renaming doesn't cause libraries not to be found.

Some advice on the best approach to fixing this would be appreciated - many thanks.

spl237 commented Mar 9, 2016

Eric, how many libraries are likely to have this problem of two copies installed? Which libraries does mesa / your driver add which are duplicates of existing VC libraries? I'm assuming this won't be the only one; whatever fix we come up with may as well deal with all of them.

We can't remove the old libraries; they are needed if the new driver is disabled, which it is by default. Is the right thing to do to adjust LD_LIBRARY_PATH on the system to change the order in which /opt/vc/lib and /usr/lib/arm-linux-gnueabihf appear depending on whether or not the new driver is enabled? If so, we can add that to raspi-config; is that going to be enough to fix this? Of course, that may also have unintended side effects elsewhere - I don't know what other applications are going to do based on a change in the order of the library path.

The alternative is to overwrite the libraries in /opt/vc/lib with the new ones, keeping backup copies; I have no idea what else that might break in the process.

Or would it be better to actually rename the new libraries completely so that both sets can coexist on the system? That strikes me as the cleanest approach and the least likely to cause longer-term problems - maybe with a relevant symlink being set up so that the renaming doesn't cause libraries not to be found.

Some advice on the best approach to fixing this would be appreciated - many thanks.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Mar 30, 2016

@anholt @spl237
Any updates on this one? JOGL (OpenGL for Java) has been broken for two month on Raspbian now :(

gohai commented Mar 30, 2016

@anholt @spl237
Any updates on this one? JOGL (OpenGL for Java) has been broken for two month on Raspbian now :(

@anholt

This comment has been minimized.

Show comment
Hide comment
@anholt

anholt Apr 14, 2016

Owner

@spl237 Yeah, if you're relying on raspi-config to enable the driver, then also editing ld.so.conf sounds good. The raspi-config also needs to be setting enable_warnings=2 for the driver to work, which is apparently not happening yet according to many users.

Owner

anholt commented Apr 14, 2016

@spl237 Yeah, if you're relying on raspi-config to enable the driver, then also editing ld.so.conf sounds good. The raspi-config also needs to be setting enable_warnings=2 for the driver to work, which is apparently not happening yet according to many users.

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Apr 14, 2016

@spl237 https://github.com/spl237 Yeah, if you're relying on raspi-config to enable the driver, then also editing ld.so.conf sounds good.

Can you please tell me exactly what needs to be changed in ld.so.conf? What needs to be set in that file in both cases; running the original driver and running your new driver?

The raspi-config also needs to be setting enable_warnings=2 for the driver to work, which is apparently not happening yet according to many users.

Where does it need to set this? It doesn’t look like an environment variable - which file does it need to go in?

Many thanks,

S.

spl237 commented Apr 14, 2016

@spl237 https://github.com/spl237 Yeah, if you're relying on raspi-config to enable the driver, then also editing ld.so.conf sounds good.

Can you please tell me exactly what needs to be changed in ld.so.conf? What needs to be set in that file in both cases; running the original driver and running your new driver?

The raspi-config also needs to be setting enable_warnings=2 for the driver to work, which is apparently not happening yet according to many users.

Where does it need to set this? It doesn’t look like an environment variable - which file does it need to go in?

Many thanks,

S.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Apr 14, 2016

I am just a pedestrian, but if I could think out loud:

If the original driver would also exist with a soname of libGLESv2.so.2, matching the Mesa one, then simply the ordering of files inside of /etc/ld.so.conf.d would be sufficient to "select" on over the other for us. In my understanding (and I might be wrong), if 00-vmcs.conf (which pulls in /opt/vc/lib), comes before arm-linux-gnueabihf.conf, we'd be loading the old driver - and vice versa.

Would it be possible to:

  • ship a copy of the old driver with SONAME set to libGLESv2.so.2
  • move 00-vmcs.conf ahead or behind arm-linux-gnueabihf.conf in raspi-config, depending on which driver is selected (an alternative would be to remove 00-vcms.conf for the vc4 case - unsure how much of the rest of /opt/vc/lib continues to works in this scenario anyway)

Note: just modifying the order in /etc/ld.so.conf.d, without the other change, wouldn't work for JOGL, since it will always first try to load libGLESv2.so.2 before falling back to libGLESv2.so.

@spl237 It's a setting in /boot/config.txt - I believe Eric meant avoid_warnings=2.

gohai commented Apr 14, 2016

I am just a pedestrian, but if I could think out loud:

If the original driver would also exist with a soname of libGLESv2.so.2, matching the Mesa one, then simply the ordering of files inside of /etc/ld.so.conf.d would be sufficient to "select" on over the other for us. In my understanding (and I might be wrong), if 00-vmcs.conf (which pulls in /opt/vc/lib), comes before arm-linux-gnueabihf.conf, we'd be loading the old driver - and vice versa.

Would it be possible to:

  • ship a copy of the old driver with SONAME set to libGLESv2.so.2
  • move 00-vmcs.conf ahead or behind arm-linux-gnueabihf.conf in raspi-config, depending on which driver is selected (an alternative would be to remove 00-vcms.conf for the vc4 case - unsure how much of the rest of /opt/vc/lib continues to works in this scenario anyway)

Note: just modifying the order in /etc/ld.so.conf.d, without the other change, wouldn't work for JOGL, since it will always first try to load libGLESv2.so.2 before falling back to libGLESv2.so.

@spl237 It's a setting in /boot/config.txt - I believe Eric meant avoid_warnings=2.

@anholt

This comment has been minimized.

Show comment
Hide comment
@anholt

anholt Apr 15, 2016

Owner

Yeah, avoid_warnings. Sorry.

Owner

anholt commented Apr 15, 2016

Yeah, avoid_warnings. Sorry.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Apr 15, 2016

@anholt Does the rest (proposed scheme) seem sensible to you? I am really no expert in all this - just hoping to get Processing to run, which - like the JOGL library it builds upon for 3D - tries to be portable. Thanks!

gohai commented Apr 15, 2016

@anholt Does the rest (proposed scheme) seem sensible to you? I am really no expert in all this - just hoping to get Processing to run, which - like the JOGL library it builds upon for 3D - tries to be portable. Thanks!

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Jun 8, 2016

Gottfried, can I please ask about your comment above:

JOGL will ask the linker first for libGLESv2.so.2 (and would later fall back to libGLESv2.so) - so the Mesa one gets loaded, even though we're on BC IV.

Is it a deliberate decision on your part to ask for libGLESv2.so.2 rather than just to ask for libGLESv2.so? If so, why?

spl237 commented Jun 8, 2016

Gottfried, can I please ask about your comment above:

JOGL will ask the linker first for libGLESv2.so.2 (and would later fall back to libGLESv2.so) - so the Mesa one gets loaded, even though we're on BC IV.

Is it a deliberate decision on your part to ask for libGLESv2.so.2 rather than just to ask for libGLESv2.so? If so, why?

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Jun 8, 2016

My only source for this was the code that determines the loading: EGLES2DynamicLibraryBundleInfo.java#L65
(pinging @sgothel @xranby)

But thinking aloud: if JOGL would request libGLESv2.so instead of .2, then we'd still be getting the "wrong" library when running one of the drivers, unless the LD_LIBRARY_PATH order gets adjusted based on which driver is selected?

gohai commented Jun 8, 2016

My only source for this was the code that determines the loading: EGLES2DynamicLibraryBundleInfo.java#L65
(pinging @sgothel @xranby)

But thinking aloud: if JOGL would request libGLESv2.so instead of .2, then we'd still be getting the "wrong" library when running one of the drivers, unless the LD_LIBRARY_PATH order gets adjusted based on which driver is selected?

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Jun 8, 2016

But thinking aloud: if JOGL would request libGLESv2.so instead of .2, then we'd still be getting the "wrong" library when running one of the drivers, unless the LD_LIBRARY_PATH order gets adjusted based on which driver is selected?

Yes, exactly - we'd like to fix this with as few changes to existing libraries and locations as possible - if you can just use the library which is first on the path with the name you require, we can simply reorder the library path depending on which driver is in use.

spl237 commented Jun 8, 2016

But thinking aloud: if JOGL would request libGLESv2.so instead of .2, then we'd still be getting the "wrong" library when running one of the drivers, unless the LD_LIBRARY_PATH order gets adjusted based on which driver is selected?

Yes, exactly - we'd like to fix this with as few changes to existing libraries and locations as possible - if you can just use the library which is first on the path with the name you require, we can simply reorder the library path depending on which driver is in use.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Jun 8, 2016

Great!

Regarding the .2: I understand that you wouldn't want to change the location or SONAME of the binary driver at this point - but have you considered ways to making it also accessible through the libGLESv2.so.2 name? Hope that Sven or Xerxes can tell us more about the preference for the latter.

gohai commented Jun 8, 2016

Great!

Regarding the .2: I understand that you wouldn't want to change the location or SONAME of the binary driver at this point - but have you considered ways to making it also accessible through the libGLESv2.so.2 name? Hope that Sven or Xerxes can tell us more about the preference for the latter.

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Jun 8, 2016

Yes, we've considered that, but it seems that spoofing ldconfig isn't easy - whenever ldconfig is run, it creates symlinks according to the sonames in the actual libraries; creating symlinks with the wrong names doesn't seem to work, or at least it didn't when I tried it this morning...

spl237 commented Jun 8, 2016

Yes, we've considered that, but it seems that spoofing ldconfig isn't easy - whenever ldconfig is run, it creates symlinks according to the sonames in the actual libraries; creating symlinks with the wrong names doesn't seem to work, or at least it didn't when I tried it this morning...

@xranby

This comment has been minimized.

Show comment
Hide comment
@xranby

xranby Jun 8, 2016

Ubuntu's EGL Driver Packaging HOWTO is designed to solve this kind of vendor driver packaging issue on multiarch systems.
https://wiki.ubuntu.com/X/EGLDriverPackagingHOWTO

Assumptions

The selection of EGL implementation determines which GL|ES and OpenVG implementation is usable. For example, it would never be useful to use a vendor's GLESv2 library with Mesa's EGL.

The libraries (with SONAME) are libEGL.so.1, libGLESv1_CM.so.1, libGLESv2.so.2 and libOpenVG.so.1

Summary

A vendor's libraries must be installed into a private library path. This will typically be of the form /usr/lib/$VENDOR-egl or /usr/lib/$DEB_HOST_MULTIARCH/$VENDOR-egl.

The vendor packages must include an ld.so.conf file in their private library path. This file should consist of a single line of text, containing the private library path as an absolute path. This means the ld.so.conf file will typically contain a path of the form 

/usr/lib/$VENDOR-egl

or

/usr/lib/$DEB_HOST_MULTIARCH/$VENDOR-egl

The vendor packages must add an ${DEB_HOST_MULTIARCH}_EGL_conf alternative to their ld.so.conf snippet in their postinst maintainer script and remove this alternative in their prerm maintainer script. The alternative should be added at a higher priority than the Mesa alternative (which has a priority of 500).

A package using this alternatives scheme must not Provide: any of the libegl1-x11, libgles1, libgles2 or libopenvg1 virtual packages.

A vendor's EGL/GL|ES/OpenVG implementation should be contained in a single package. There is no facility for resolving application dependencies to specific packages in a vendor's implementation. If the vendor also ships an Xorg DDX and/or OpenGL implementation, these may all be shipped as a single package. Note however that OpenGL implementations require additional integration work not covered in this document. 

xranby commented Jun 8, 2016

Ubuntu's EGL Driver Packaging HOWTO is designed to solve this kind of vendor driver packaging issue on multiarch systems.
https://wiki.ubuntu.com/X/EGLDriverPackagingHOWTO

Assumptions

The selection of EGL implementation determines which GL|ES and OpenVG implementation is usable. For example, it would never be useful to use a vendor's GLESv2 library with Mesa's EGL.

The libraries (with SONAME) are libEGL.so.1, libGLESv1_CM.so.1, libGLESv2.so.2 and libOpenVG.so.1

Summary

A vendor's libraries must be installed into a private library path. This will typically be of the form /usr/lib/$VENDOR-egl or /usr/lib/$DEB_HOST_MULTIARCH/$VENDOR-egl.

The vendor packages must include an ld.so.conf file in their private library path. This file should consist of a single line of text, containing the private library path as an absolute path. This means the ld.so.conf file will typically contain a path of the form 

/usr/lib/$VENDOR-egl

or

/usr/lib/$DEB_HOST_MULTIARCH/$VENDOR-egl

The vendor packages must add an ${DEB_HOST_MULTIARCH}_EGL_conf alternative to their ld.so.conf snippet in their postinst maintainer script and remove this alternative in their prerm maintainer script. The alternative should be added at a higher priority than the Mesa alternative (which has a priority of 500).

A package using this alternatives scheme must not Provide: any of the libegl1-x11, libgles1, libgles2 or libopenvg1 virtual packages.

A vendor's EGL/GL|ES/OpenVG implementation should be contained in a single package. There is no facility for resolving application dependencies to specific packages in a vendor's implementation. If the vendor also ships an Xorg DDX and/or OpenGL implementation, these may all be shipped as a single package. Note however that OpenGL implementations require additional integration work not covered in this document. 
@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Jun 8, 2016

Xerxes, we already do that - the VideoCore GL libraries are all installed in /opt/vc/lib, and the Mesa libraries are in /usr/lib/arm-linux-gnueabihf/ We would therefore be able to switch between the VC and the Mesa libraries just by changing the settings in ld.so.conf, but that doesn't work in this case because JOGL looks for (and finds) libGLESv2.so.2 (the Mesa version) in preference to libGLESv2.so (the VC version) - this is the part of the mechanism that I don't understand. We can potentially change the soname of the VC library to end with the .so.2 extension to make it match that of the Mesa version, in which case switching with ld.conf should work, but I don't know if that is the only way to fix this issue.

spl237 commented Jun 8, 2016

Xerxes, we already do that - the VideoCore GL libraries are all installed in /opt/vc/lib, and the Mesa libraries are in /usr/lib/arm-linux-gnueabihf/ We would therefore be able to switch between the VC and the Mesa libraries just by changing the settings in ld.so.conf, but that doesn't work in this case because JOGL looks for (and finds) libGLESv2.so.2 (the Mesa version) in preference to libGLESv2.so (the VC version) - this is the part of the mechanism that I don't understand. We can potentially change the soname of the VC library to end with the .so.2 extension to make it match that of the Mesa version, in which case switching with ld.conf should work, but I don't know if that is the only way to fix this issue.

@xranby

This comment has been minimized.

Show comment
Hide comment
@xranby

xranby Jun 8, 2016

The intention when using the alternative system is to only have one set of opengl es drivers available to the linker at a given time:

The VideoCore provided 00-vmcs.conf must be moved from
/etc/ld.so.conf.d/00-vmcs.conf to /opt/vc/lib/
for this to work.

To summarize
/etc/ld.so.conf.d shall contain only one symlink that points to the alternatives symlink, example like this:
/etc/ld.so.conf.d/armhf-linux-gnueabi_EGL.conf -> /etc/alternatives/armhf-linux-gnueabi_egl_conf

The /etc/alternatives/armhf-linux-gnueabi_egl_conf symlink is to be manipulated using the alternatives system "update-alternatives" command and shall point to the active driver implementations ld.so.conf file

Examples
to use the Mesa3D implementation, then run:
sudo update-alternatives --set armhf-linux-gnueabi_egl_conf /usr/lib/armhf-linux-gnueabi/mesa-egl/ld.so.conf
to use the VC implementation then run:
sudo update-alternatives --set armhf-linux-gnueabi_egl_conf /opt/vc/lib/00-vmcs.conf

xranby commented Jun 8, 2016

The intention when using the alternative system is to only have one set of opengl es drivers available to the linker at a given time:

The VideoCore provided 00-vmcs.conf must be moved from
/etc/ld.so.conf.d/00-vmcs.conf to /opt/vc/lib/
for this to work.

To summarize
/etc/ld.so.conf.d shall contain only one symlink that points to the alternatives symlink, example like this:
/etc/ld.so.conf.d/armhf-linux-gnueabi_EGL.conf -> /etc/alternatives/armhf-linux-gnueabi_egl_conf

The /etc/alternatives/armhf-linux-gnueabi_egl_conf symlink is to be manipulated using the alternatives system "update-alternatives" command and shall point to the active driver implementations ld.so.conf file

Examples
to use the Mesa3D implementation, then run:
sudo update-alternatives --set armhf-linux-gnueabi_egl_conf /usr/lib/armhf-linux-gnueabi/mesa-egl/ld.so.conf
to use the VC implementation then run:
sudo update-alternatives --set armhf-linux-gnueabi_egl_conf /opt/vc/lib/00-vmcs.conf

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Jun 8, 2016

It's not as simple as that. The 00-vmcs.conf symlink is required at all times as otherwise the non-GL VC specific libraries won't be found, and they share a directory with the VC-specific GL libraries. As do the Mesa GL libraries with numerous other non-GL libraries. The mechanism above would only work if each set of GL libraries were installed in completely isolated directories, which is not the case at present for either Mesa or VC.

We're going to try adding the version number prefixes to the VC GL libraries, and then use reordering of search paths in ld.so.conf to prioritise one over the other.

spl237 commented Jun 8, 2016

It's not as simple as that. The 00-vmcs.conf symlink is required at all times as otherwise the non-GL VC specific libraries won't be found, and they share a directory with the VC-specific GL libraries. As do the Mesa GL libraries with numerous other non-GL libraries. The mechanism above would only work if each set of GL libraries were installed in completely isolated directories, which is not the case at present for either Mesa or VC.

We're going to try adding the version number prefixes to the VC GL libraries, and then use reordering of search paths in ld.so.conf to prioritise one over the other.

@xranby

This comment has been minimized.

Show comment
Hide comment
@xranby

xranby Jun 8, 2016

Adding the version number prefixes to EGL and GLESv2 is in line with how Debian [1] reads the Khronos Implementers Guide [2] and OpenGL Application Binary Interface for Linux [3] specification. All recommended.

It is worth mentioning that both Debian and Ubuntu diverge from the OpenGL Application Binary Interface for Linux [3] specification. The OpenGL Application Binary Interface for Linux [3] made it impossible to have two implementations on the same system by mandating that the implementation must be located in /usr/lib

To quote the specification [3]:
"3.2. These libraries must be located in /usr/lib. The X-specific library direction (/usr/lib/X11) was also considered, but existing practice on Linux and other platforms indicates that /usr/lib is preferable. "

As can be seen in Debian file list the EGL libs are placed in the /usr/lib/multiarch location
https://packages.debian.org/jessie/armhf/libegl1-mesa/filelist
On Ubuntu the EGL libs are placed in the /usr/lib/multiarch/alternatives location
http://packages.ubuntu.com/xenial/armhf/libegl1-mesa/filelist

[1] https://anonscm.debian.org/cgit/pkg-xorg/lib/mesa.git/tree/debian/README.Debian
[2] https://www.khronos.org/registry/implementers_guide.html#idm58020503456
[3] https://www.opengl.org/registry/ABI/#3

xranby commented Jun 8, 2016

Adding the version number prefixes to EGL and GLESv2 is in line with how Debian [1] reads the Khronos Implementers Guide [2] and OpenGL Application Binary Interface for Linux [3] specification. All recommended.

It is worth mentioning that both Debian and Ubuntu diverge from the OpenGL Application Binary Interface for Linux [3] specification. The OpenGL Application Binary Interface for Linux [3] made it impossible to have two implementations on the same system by mandating that the implementation must be located in /usr/lib

To quote the specification [3]:
"3.2. These libraries must be located in /usr/lib. The X-specific library direction (/usr/lib/X11) was also considered, but existing practice on Linux and other platforms indicates that /usr/lib is preferable. "

As can be seen in Debian file list the EGL libs are placed in the /usr/lib/multiarch location
https://packages.debian.org/jessie/armhf/libegl1-mesa/filelist
On Ubuntu the EGL libs are placed in the /usr/lib/multiarch/alternatives location
http://packages.ubuntu.com/xenial/armhf/libegl1-mesa/filelist

[1] https://anonscm.debian.org/cgit/pkg-xorg/lib/mesa.git/tree/debian/README.Debian
[2] https://www.khronos.org/registry/implementers_guide.html#idm58020503456
[3] https://www.opengl.org/registry/ABI/#3

@spl237

This comment has been minimized.

Show comment
Hide comment
@spl237

spl237 Jun 9, 2016

We've regenerated our libraries so the sonames match those specified in the Ubuntu documentation, and have modified raspi-config so that it changes the order of directories used in ldconf; this certainly fixes the gstreamer issue mentioned above, and does not appear to have broken anything else. We'll do some more testing over the next few weeks and push the changes into rpi-update if everything looks good.

spl237 commented Jun 9, 2016

We've regenerated our libraries so the sonames match those specified in the Ubuntu documentation, and have modified raspi-config so that it changes the order of directories used in ldconf; this certainly fixes the gstreamer issue mentioned above, and does not appear to have broken anything else. We'll do some more testing over the next few weeks and push the changes into rpi-update if everything looks good.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Jun 9, 2016

@spl237 Awesome. Can't wait to test!

gohai commented Jun 9, 2016

@spl237 Awesome. Can't wait to test!

popcornmix added a commit to Hexxeh/rpi-firmware that referenced this issue Jul 2, 2016

popcornmix added a commit to raspberrypi/firmware that referenced this issue Jul 2, 2016

popcornmix added a commit to raspberrypi/userland that referenced this issue Jul 2, 2016

@Lekensteyn

This comment has been minimized.

Show comment
Hide comment
@Lekensteyn

Lekensteyn Jul 3, 2016

@popcornmix and others, the latest commit (1f8d685c22761b9a5ee3343b01e79d31e2ed0f6b) breaks all applications linking with libEGL.so.1 on Arch Linux ARM.

$ rrdtool --help
rrdtool: symbol lookup error: /opt/vc/lib/libEGL.so.1: undefined symbol: glDiscardFramebufferEXT

Reverting to the hardfp/lib/ files from the previous commit (65e34b54919dcc1e3824c99fe0432b10212c92f2) fixes the issue. Ensuring use of mesa's libEGL.so.1 (rename libEGL.so.1 to libEGL.so for vc) works too. Just FYI...

@popcornmix and others, the latest commit (1f8d685c22761b9a5ee3343b01e79d31e2ed0f6b) breaks all applications linking with libEGL.so.1 on Arch Linux ARM.

$ rrdtool --help
rrdtool: symbol lookup error: /opt/vc/lib/libEGL.so.1: undefined symbol: glDiscardFramebufferEXT

Reverting to the hardfp/lib/ files from the previous commit (65e34b54919dcc1e3824c99fe0432b10212c92f2) fixes the issue. Ensuring use of mesa's libEGL.so.1 (rename libEGL.so.1 to libEGL.so for vc) works too. Just FYI...

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 3, 2016

Have you relinked the application since updating with rpi-update?

Have you relinked the application since updating with rpi-update?

@Lekensteyn

This comment has been minimized.

Show comment
Hide comment
@Lekensteyn

Lekensteyn Jul 3, 2016

rrdtool is not using EGL AFAIK, it is pulled in because some other library depends on libEGL AFAIK.

Even if I relink, what other shared library would provide this symbol? It is certainly not libEGL.so :/

$ nm lib/libEGL.so | grep glDiscardFramebufferEXT
         U glDiscardFramebufferEXT

Similar reports from other distros:
https://lists.opensuse.org/opensuse-arm/2016-01/msg00015.html

rrdtool is not using EGL AFAIK, it is pulled in because some other library depends on libEGL AFAIK.

Even if I relink, what other shared library would provide this symbol? It is certainly not libEGL.so :/

$ nm lib/libEGL.so | grep glDiscardFramebufferEXT
         U glDiscardFramebufferEXT

Similar reports from other distros:
https://lists.opensuse.org/opensuse-arm/2016-01/msg00015.html

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 3, 2016

What does ldconfig -v 2>/dev/null | grep -v ^$'\t' report?
The idea is that after the /opt/vc/lib change, ordering the library path should ensure you get the libEGL.so from the library first on the library path.

What does ldconfig -v 2>/dev/null | grep -v ^$'\t' report?
The idea is that after the /opt/vc/lib change, ordering the library path should ensure you get the libEGL.so from the library first on the library path.

@Lekensteyn

This comment has been minimized.

Show comment
Hide comment
@Lekensteyn

Lekensteyn Jul 3, 2016

The output is:

/opt/vc/lib:
/usr/lib/libfakeroot:
/usr/lib:

because of:

$ grep -r . /etc/ld.so.conf*
/etc/ld.so.conf:#
/etc/ld.so.conf:# /etc/ld.so.conf
/etc/ld.so.conf:#
/etc/ld.so.conf:include /etc/ld.so.conf.d/*.conf
/etc/ld.so.conf:# End of file
/etc/ld.so.conf:include /etc/ld.so.conf.d/*.conf
/etc/ld.so.conf.d/fakeroot.conf:/usr/lib/libfakeroot
/etc/ld.so.conf.d/00-raspberrypi-firmware.conf:/opt/vc/lib/

The package is constructed like this:
https://archlinuxarm.org/packages/any/raspberrypi-firmware-tools/files/PKGBUILD

Should it (the /opt/vc/lib/ entry) be moved to the end (of the ld.so.conf sequence)?
NOTE: libGL.so from mesa works, the VC one breaks.

Lekensteyn commented Jul 3, 2016

The output is:

/opt/vc/lib:
/usr/lib/libfakeroot:
/usr/lib:

because of:

$ grep -r . /etc/ld.so.conf*
/etc/ld.so.conf:#
/etc/ld.so.conf:# /etc/ld.so.conf
/etc/ld.so.conf:#
/etc/ld.so.conf:include /etc/ld.so.conf.d/*.conf
/etc/ld.so.conf:# End of file
/etc/ld.so.conf:include /etc/ld.so.conf.d/*.conf
/etc/ld.so.conf.d/fakeroot.conf:/usr/lib/libfakeroot
/etc/ld.so.conf.d/00-raspberrypi-firmware.conf:/opt/vc/lib/

The package is constructed like this:
https://archlinuxarm.org/packages/any/raspberrypi-firmware-tools/files/PKGBUILD

Should it (the /opt/vc/lib/ entry) be moved to the end (of the ld.so.conf sequence)?
NOTE: libGL.so from mesa works, the VC one breaks.

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 3, 2016

Can you be clear which libEGL you want? The mesa or /opt/vc one?
Is the mesa one in /usr/lib? (it is in /usr/lib/arm-linux-gnueabihf on raspbian).
Yes, you want your ld library path to find the preferred lib first.
The idea is that a tool like raspi-config can switch the ordering of the library to switch between the /opt/vc libs and the mesa libs.

Can you be clear which libEGL you want? The mesa or /opt/vc one?
Is the mesa one in /usr/lib? (it is in /usr/lib/arm-linux-gnueabihf on raspbian).
Yes, you want your ld library path to find the preferred lib first.
The idea is that a tool like raspi-config can switch the ordering of the library to switch between the /opt/vc libs and the mesa libs.

@Lekensteyn

This comment has been minimized.

Show comment
Hide comment
@Lekensteyn

Lekensteyn Jul 3, 2016

Normally Mesa's libEGL is wanted (/usr/lib/libEGL.so* -> /usr/lib/mesa/libEGL.so.1.0.0), presumably /opt/vc/lib/ is present in the search path for binaries in /opt/vc/bin/.

If one switches the search path (e.g. with raspi-config), wouldn't that break all applications that rely on Mesa's libEGL? The problem now is that loading libEGL.so does not exactly work (because of the missing symbol, see above). Any idea why the symbol is undefined?

Normally Mesa's libEGL is wanted (/usr/lib/libEGL.so* -> /usr/lib/mesa/libEGL.so.1.0.0), presumably /opt/vc/lib/ is present in the search path for binaries in /opt/vc/bin/.

If one switches the search path (e.g. with raspi-config), wouldn't that break all applications that rely on Mesa's libEGL? The problem now is that loading libEGL.so does not exactly work (because of the missing symbol, see above). Any idea why the symbol is undefined?

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 3, 2016

You can't run both libs concurrently. You need to modify config.txt (dtoverlay=vc4-kms-v3d) and reboot to use the mesa libs and remove it and reboot to use the /opt/vc libs.
The idea is that when modifying config.txt you also modify the ld library path.

Note that the mesa and /opt/vc libs are not plug-in replacements. Code has to be written for one or the other (use of the /opt/vc libs requires dispmanx setup code to make the output visible). The list of symbols in each library is significantly different too.

If you are using mesa libs then does ensuring /opt/vc/lib is later on the library path fix your issue?

You can't run both libs concurrently. You need to modify config.txt (dtoverlay=vc4-kms-v3d) and reboot to use the mesa libs and remove it and reboot to use the /opt/vc libs.
The idea is that when modifying config.txt you also modify the ld library path.

Note that the mesa and /opt/vc libs are not plug-in replacements. Code has to be written for one or the other (use of the /opt/vc libs requires dispmanx setup code to make the output visible). The list of symbols in each library is significantly different too.

If you are using mesa libs then does ensuring /opt/vc/lib is later on the library path fix your issue?

@Lekensteyn

This comment has been minimized.

Show comment
Hide comment
@Lekensteyn

Lekensteyn Jul 3, 2016

I haven't tried moving /opt/vc/lib/ later in the search path as preventing libEGL.so.1 from appearing in the search path also results in mesa's libEGL.so.1 being found and used. So yes, this "fixes" the issue :-)

FWIW, I do not need (E)GL functionality, this RPi is running headless. But programs that link libEGL.so anyway become quite upset if the wrong library is being used.

I haven't tried moving /opt/vc/lib/ later in the search path as preventing libEGL.so.1 from appearing in the search path also results in mesa's libEGL.so.1 being found and used. So yes, this "fixes" the issue :-)

FWIW, I do not need (E)GL functionality, this RPi is running headless. But programs that link libEGL.so anyway become quite upset if the wrong library is being used.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Jul 7, 2016

@popcornmix @spl237
Now that the SONAME change got reverted again, what's your current thinking on the matter? The current behavior needs to stay, with both libraries available at all times, irrespective of what driver we're running?

gohai commented Jul 7, 2016

@popcornmix @spl237
Now that the SONAME change got reverted again, what's your current thinking on the matter? The current behavior needs to stay, with both libraries available at all times, irrespective of what driver we're running?

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 7, 2016

Latest update reverts the SOVERSION change and now adds additional libs: libbrcmEGL.so, libbrcmGLESv2.so, libbrcmOpenVG.so and libbrcmWFC.so.
Both sets of libs will exist for some time (e.g. two months), and applications will be encouraged to switch to the "brcm" naming of the graphics libs.
A future update will remove the previous named versions of the libs, and at that point we should have no conflicts.

Latest update reverts the SOVERSION change and now adds additional libs: libbrcmEGL.so, libbrcmGLESv2.so, libbrcmOpenVG.so and libbrcmWFC.so.
Both sets of libs will exist for some time (e.g. two months), and applications will be encouraged to switch to the "brcm" naming of the graphics libs.
A future update will remove the previous named versions of the libs, and at that point we should have no conflicts.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Jul 7, 2016

Thanks for the update @popcornmix. I can just say from the perspective of an app that should be able to work with both GLES implementations: for us the proposed change (renaming the Broadcom library) is pretty much equivalent with the original situation of having two libraries with different SONAMES. In both cases, the linker won't hand us the correct implementation depending on which driver is currently loaded. Instead, the only alternative that I see is: detect somehow that we're running on a Raspberry Pi, with the VC4 DRM driver not loaded, and in this case we'd have to attempt to load libbrcmGLESv2.so, and do the dispmanx-specific init.

So if the switch to libbrcm will cause some headaches for other people: please don't do it for the JOGL or the Processing project, because (as far as I understand the situation) it wouldn't make much of a difference there.

gohai commented Jul 7, 2016

Thanks for the update @popcornmix. I can just say from the perspective of an app that should be able to work with both GLES implementations: for us the proposed change (renaming the Broadcom library) is pretty much equivalent with the original situation of having two libraries with different SONAMES. In both cases, the linker won't hand us the correct implementation depending on which driver is currently loaded. Instead, the only alternative that I see is: detect somehow that we're running on a Raspberry Pi, with the VC4 DRM driver not loaded, and in this case we'd have to attempt to load libbrcmGLESv2.so, and do the dispmanx-specific init.

So if the switch to libbrcm will cause some headaches for other people: please don't do it for the JOGL or the Processing project, because (as far as I understand the situation) it wouldn't make much of a difference there.

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 7, 2016

@anholt what is your view? Is the latest update to firmware libs worthwhile for you?
There will be a fair amount of pain when we finally remove /opt/vc/lib/libEGL.so etc, so we'll only do it if it helps some users.

@anholt what is your view? Is the latest update to firmware libs worthwhile for you?
There will be a fair amount of pain when we finally remove /opt/vc/lib/libEGL.so etc, so we'll only do it if it helps some users.

@loganmc10

This comment has been minimized.

Show comment
Hide comment
@loganmc10

loganmc10 Jul 7, 2016

This will break SDL2:

http://hg.libsdl.org/SDL/file/a6ad6813b17b/src/video/SDL_egl.c#l41

They currently hardcode the path to those libraries (for the firmware driver).

This will break SDL2:

http://hg.libsdl.org/SDL/file/a6ad6813b17b/src/video/SDL_egl.c#l41

They currently hardcode the path to those libraries (for the firmware driver).

@popcornmix

This comment has been minimized.

Show comment
Hide comment
@popcornmix

popcornmix Jul 7, 2016

Yes (not currently but when /opt/vc/lib/libEGL.so is removed).
But all apps that use firmware libEGL.so will need rebuilding.

Hard coding /opt/vc/lib is not ideal. /opt/vc/lib isn't the install path used by all distributions.

Yes (not currently but when /opt/vc/lib/libEGL.so is removed).
But all apps that use firmware libEGL.so will need rebuilding.

Hard coding /opt/vc/lib is not ideal. /opt/vc/lib isn't the install path used by all distributions.

@anholt

This comment has been minimized.

Show comment
Hide comment
@anholt

anholt Jul 14, 2016

Owner

It seems like having the brcm libs matching in SONAME with the Mesa libs could be useful, as long as boot-time scripts make sure that ld.so.conf is set to point to whichever driver is loaded at this boot. Then apps could link just against libEGL/libGLESv2 like normal, and use https://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_platform_base.txt and EGL_EXT_platform_x11 extensions to decide if the GL implementation under them supports normal X11 initialization (open driver), or if dispmanx display init should be called.

The undefined symbol issue earlier sure sounds like you had something like "we used the brcm libEGL but the Mesa libGLES was in the path and brcm libEGL tried to use that and failed".

Owner

anholt commented Jul 14, 2016

It seems like having the brcm libs matching in SONAME with the Mesa libs could be useful, as long as boot-time scripts make sure that ld.so.conf is set to point to whichever driver is loaded at this boot. Then apps could link just against libEGL/libGLESv2 like normal, and use https://www.khronos.org/registry/egl/extensions/EXT/EGL_EXT_platform_base.txt and EGL_EXT_platform_x11 extensions to decide if the GL implementation under them supports normal X11 initialization (open driver), or if dispmanx display init should be called.

The undefined symbol issue earlier sure sounds like you had something like "we used the brcm libEGL but the Mesa libGLES was in the path and brcm libEGL tried to use that and failed".

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Mar 14, 2017

Is there a good way for detecting that the system is running a Fake-KMS+Mesa type configuration?

The custom workarounds for Processing we put into JOGL to support both stacks currently looks for the presence of the vc4 kernel module as a trigger to not do the dispmanx dance. Will this still work with Fake-KMS?

gohai commented Mar 14, 2017

Is there a good way for detecting that the system is running a Fake-KMS+Mesa type configuration?

The custom workarounds for Processing we put into JOGL to support both stacks currently looks for the presence of the vc4 kernel module as a trigger to not do the dispmanx dance. Will this still work with Fake-KMS?

@anholt

This comment has been minimized.

Show comment
Hide comment
@anholt

anholt Mar 15, 2017

Owner

Yeah, vc4 will still be present, and you get to use vc4 just like you would in full kms mode.

Owner

anholt commented Mar 15, 2017

Yeah, vc4 will still be present, and you get to use vc4 just like you would in full kms mode.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Mar 15, 2017

That's great news.

I am closing this bug: In Processing we'll be shipping a custom version of JOGL, that - depending on whether we detect the vc4 module to be present - will change the order in which we attempt to load libGLESv2, and also enable or disable the special code paths for the GLES2 driver.

I wish there would have been a solution that wouldn't have pushed this detection logic into each application, but well. We're still very excited for the Mesa driver!

gohai commented Mar 15, 2017

That's great news.

I am closing this bug: In Processing we'll be shipping a custom version of JOGL, that - depending on whether we detect the vc4 module to be present - will change the order in which we attempt to load libGLESv2, and also enable or disable the special code paths for the GLES2 driver.

I wish there would have been a solution that wouldn't have pushed this detection logic into each application, but well. We're still very excited for the Mesa driver!

@gohai gohai closed this Mar 15, 2017

@gouessej

This comment has been minimized.

Show comment
Hide comment
@gouessej

gouessej Mar 22, 2017

@gohai I'd like to see your fixes in JOGL 2.3.3 so that other JOGL users benefit of it.

@gohai I'd like to see your fixes in JOGL 2.3.3 so that other JOGL users benefit of it.

@gohai

This comment has been minimized.

Show comment
Hide comment
@gohai

gohai Mar 22, 2017

@gouessej I've sent PRs for all of them (first one in Feb 2016), but this acted upon, at least not so far...

gohai commented Mar 22, 2017

@gouessej I've sent PRs for all of them (first one in Feb 2016), but this acted upon, at least not so far...

anholt pushed a commit that referenced this issue May 2, 2017

util/queue: don't hang at exit
So atexit() is horrible and 4aea8fe is probably not a good idea.  But
add an extra layer of duct-tape to the problem.  Otherwise we hit a
situation where app using an atexit() handler that runs later than ours
doesn't hang when trying to tear down a context.

 (gdb) bt
 #0  util_queue_killall_and_wait (queue=queue@entry=0x52bc80) at ../../../src/util/u_queue.c:264
 #1  0x0000007fb6c380c0 in atexit_handler () at ../../../src/util/u_queue.c:51
 #2  0x0000007fb7730e2c in __run_exit_handlers () from /lib64/libc.so.6
 #3  0x0000007fb7730e5c in exit () from /lib64/libc.so.6
 #4  0x0000007fb7ce17dc in piglit_report_result (result=PIGLIT_PASS) at /home/robclark/src/piglit/tests/util/piglit-util.c:267
 #5  0x0000007fb7ef99f8 in process_next_event (x11_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:139
 #6  0x0000007fb7ef9a90 in enter_event_loop (winsys_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:153
 #7  0x0000007fb7ef8e50 in run_test (gl_fw=0x432c20, argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_winsys_framework.c:88
 #8  0x0000007fb7edb890 in piglit_gl_test_run (argc=1, argv=0x7ffffff588, config=0x7ffffff400) at /home/robclark/src/piglit/tests/util/piglit-framework-gl.c:203
 #9  0x0000000000401224 in main (argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/bugs/drawbuffer-modes.c:46
 (gdb) c
 Continuing.
 [Thread 0x7fb67580c0 (LWP 3471) exited]
 ^C
 Thread 1 "drawbuffer-mode" received signal SIGINT, Interrupt.
 0x0000007fb72dda34 in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
 (gdb) bt
 #0  0x0000007fb72dda34 in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
 #1  0x0000007fb6c38304 in cnd_wait (mtx=0x5bdc90, cond=0x5bdcc0) at ../../../include/c11/threads_posix.h:159
 #2  util_queue_fence_wait (fence=0x5bdc90) at ../../../src/util/u_queue.c:106
 #3  0x0000007fb6daac70 in fd_batch_sync (batch=0x5bdc70) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:233
 #4  batch_reset (batch=batch@entry=0x5bdc70) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:183
 #5  0x0000007fb6daa5e0 in batch_flush (batch=0x5bdc70) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:290
 #6  fd_batch_flush (batch=0x5bdc70, sync=<optimized out>) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:308
 #7  0x0000007fb6daba2c in fd_bc_flush (cache=0x461220, ctx=0x52b920) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch_cache.c:141
 #8  0x0000007fb6dac954 in fd_context_flush (pctx=0x52b920, fence=0x0, flags=<optimized out>) at ../../../../../src/gallium/drivers/freedreno/freedreno_context.c:54
 #9  0x0000007fb6b43294 in st_glFlush (ctx=<optimized out>) at ../../../src/mesa/state_tracker/st_cb_flush.c:121
 #10 0x0000007fb69a84e8 in _mesa_make_current (newCtx=newCtx@entry=0x0, drawBuffer=drawBuffer@entry=0x0, readBuffer=readBuffer@entry=0x0) at ../../../src/mesa/main/context.c:1654
 #11 0x0000007fb6b7ca58 in st_api_make_current (stapi=<optimized out>, stctxi=0x0, stdrawi=0x0, streadi=0x0) at ../../../src/mesa/state_tracker/st_manager.c:827
 #12 0x0000007fb6cc87e8 in dri_unbind_context (cPriv=<optimized out>) at ../../../../../src/gallium/state_trackers/dri/dri_context.c:217
 #13 0x0000007fb6cc80b0 in driUnbindContext (pcp=0x5271e0) at ../../../../../../src/mesa/drivers/dri/common/dri_util.c:591
 #14 0x0000007fb7d1da08 in MakeContextCurrent (dpy=0x433380, draw=0, read=0, gc_user=0x0) at ../../../src/glx/glxcurrent.c:214
 #15 0x0000007fb7a8d5e0 in glx_platform_make_current () from /lib64/libwaffle-1.so.0
 #16 0x0000007fb7a894e4 in waffle_make_current () from /lib64/libwaffle-1.so.0
 #17 0x0000007fb7ef8c60 in piglit_wfl_framework_teardown (wfl_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_wfl_framework.c:628
 #18 0x0000007fb7ef939c in piglit_winsys_framework_teardown (winsys_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_winsys_framework.c:238
 #19 0x0000007fb7ef9c30 in destroy (gl_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:212
 #20 0x0000007fb7edb7c4 in destroy () at /home/robclark/src/piglit/tests/util/piglit-framework-gl.c:184
 #21 0x0000007fb7730e2c in __run_exit_handlers () from /lib64/libc.so.6
 #22 0x0000007fb7730e5c in exit () from /lib64/libc.so.6
 #23 0x0000007fb7ce17dc in piglit_report_result (result=PIGLIT_PASS) at /home/robclark/src/piglit/tests/util/piglit-util.c:267
 #24 0x0000007fb7ef99f8 in process_next_event (x11_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:139
 #25 0x0000007fb7ef9a90 in enter_event_loop (winsys_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:153
 #26 0x0000007fb7ef8e50 in run_test (gl_fw=0x432c20, argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_winsys_framework.c:88
 #27 0x0000007fb7edb890 in piglit_gl_test_run (argc=1, argv=0x7ffffff588, config=0x7ffffff400) at /home/robclark/src/piglit/tests/util/piglit-framework-gl.c:203
 #28 0x0000000000401224 in main (argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/bugs/drawbuffer-modes.c:46
 (gdb) r

Fixes: 4aea8fe ("gallium/u_queue: fix random crashes when the app calls exit()")
Signed-off-by: Rob Clark <robdclark@gmail.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>

anholt pushed a commit that referenced this issue Jun 19, 2017

util/queue: don't hang at exit
So atexit() is horrible and 4aea8fe is probably not a good idea.  But
add an extra layer of duct-tape to the problem.  Otherwise we hit a
situation where app using an atexit() handler that runs later than ours
doesn't hang when trying to tear down a context.

 (gdb) bt
 #0  util_queue_killall_and_wait (queue=queue@entry=0x52bc80) at ../../../src/util/u_queue.c:264
 #1  0x0000007fb6c380c0 in atexit_handler () at ../../../src/util/u_queue.c:51
 #2  0x0000007fb7730e2c in __run_exit_handlers () from /lib64/libc.so.6
 #3  0x0000007fb7730e5c in exit () from /lib64/libc.so.6
 #4  0x0000007fb7ce17dc in piglit_report_result (result=PIGLIT_PASS) at /home/robclark/src/piglit/tests/util/piglit-util.c:267
 #5  0x0000007fb7ef99f8 in process_next_event (x11_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:139
 #6  0x0000007fb7ef9a90 in enter_event_loop (winsys_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:153
 #7  0x0000007fb7ef8e50 in run_test (gl_fw=0x432c20, argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_winsys_framework.c:88
 #8  0x0000007fb7edb890 in piglit_gl_test_run (argc=1, argv=0x7ffffff588, config=0x7ffffff400) at /home/robclark/src/piglit/tests/util/piglit-framework-gl.c:203
 #9  0x0000000000401224 in main (argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/bugs/drawbuffer-modes.c:46
 (gdb) c
 Continuing.
 [Thread 0x7fb67580c0 (LWP 3471) exited]
 ^C
 Thread 1 "drawbuffer-mode" received signal SIGINT, Interrupt.
 0x0000007fb72dda34 in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
 (gdb) bt
 #0  0x0000007fb72dda34 in pthread_cond_wait@@GLIBC_2.17 () from /lib64/libpthread.so.0
 #1  0x0000007fb6c38304 in cnd_wait (mtx=0x5bdc90, cond=0x5bdcc0) at ../../../include/c11/threads_posix.h:159
 #2  util_queue_fence_wait (fence=0x5bdc90) at ../../../src/util/u_queue.c:106
 #3  0x0000007fb6daac70 in fd_batch_sync (batch=0x5bdc70) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:233
 #4  batch_reset (batch=batch@entry=0x5bdc70) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:183
 #5  0x0000007fb6daa5e0 in batch_flush (batch=0x5bdc70) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:290
 #6  fd_batch_flush (batch=0x5bdc70, sync=<optimized out>) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch.c:308
 #7  0x0000007fb6daba2c in fd_bc_flush (cache=0x461220, ctx=0x52b920) at ../../../../../src/gallium/drivers/freedreno/freedreno_batch_cache.c:141
 #8  0x0000007fb6dac954 in fd_context_flush (pctx=0x52b920, fence=0x0, flags=<optimized out>) at ../../../../../src/gallium/drivers/freedreno/freedreno_context.c:54
 #9  0x0000007fb6b43294 in st_glFlush (ctx=<optimized out>) at ../../../src/mesa/state_tracker/st_cb_flush.c:121
 #10 0x0000007fb69a84e8 in _mesa_make_current (newCtx=newCtx@entry=0x0, drawBuffer=drawBuffer@entry=0x0, readBuffer=readBuffer@entry=0x0) at ../../../src/mesa/main/context.c:1654
 #11 0x0000007fb6b7ca58 in st_api_make_current (stapi=<optimized out>, stctxi=0x0, stdrawi=0x0, streadi=0x0) at ../../../src/mesa/state_tracker/st_manager.c:827
 #12 0x0000007fb6cc87e8 in dri_unbind_context (cPriv=<optimized out>) at ../../../../../src/gallium/state_trackers/dri/dri_context.c:217
 #13 0x0000007fb6cc80b0 in driUnbindContext (pcp=0x5271e0) at ../../../../../../src/mesa/drivers/dri/common/dri_util.c:591
 #14 0x0000007fb7d1da08 in MakeContextCurrent (dpy=0x433380, draw=0, read=0, gc_user=0x0) at ../../../src/glx/glxcurrent.c:214
 #15 0x0000007fb7a8d5e0 in glx_platform_make_current () from /lib64/libwaffle-1.so.0
 #16 0x0000007fb7a894e4 in waffle_make_current () from /lib64/libwaffle-1.so.0
 #17 0x0000007fb7ef8c60 in piglit_wfl_framework_teardown (wfl_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_wfl_framework.c:628
 #18 0x0000007fb7ef939c in piglit_winsys_framework_teardown (winsys_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_winsys_framework.c:238
 #19 0x0000007fb7ef9c30 in destroy (gl_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:212
 #20 0x0000007fb7edb7c4 in destroy () at /home/robclark/src/piglit/tests/util/piglit-framework-gl.c:184
 #21 0x0000007fb7730e2c in __run_exit_handlers () from /lib64/libc.so.6
 #22 0x0000007fb7730e5c in exit () from /lib64/libc.so.6
 #23 0x0000007fb7ce17dc in piglit_report_result (result=PIGLIT_PASS) at /home/robclark/src/piglit/tests/util/piglit-util.c:267
 #24 0x0000007fb7ef99f8 in process_next_event (x11_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:139
 #25 0x0000007fb7ef9a90 in enter_event_loop (winsys_fw=0x432c20) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_x11_framework.c:153
 #26 0x0000007fb7ef8e50 in run_test (gl_fw=0x432c20, argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/util/piglit-framework-gl/piglit_winsys_framework.c:88
 #27 0x0000007fb7edb890 in piglit_gl_test_run (argc=1, argv=0x7ffffff588, config=0x7ffffff400) at /home/robclark/src/piglit/tests/util/piglit-framework-gl.c:203
 #28 0x0000000000401224 in main (argc=1, argv=0x7ffffff588) at /home/robclark/src/piglit/tests/bugs/drawbuffer-modes.c:46
 (gdb) r

Fixes: 4aea8fe ("gallium/u_queue: fix random crashes when the app calls exit()")
Signed-off-by: Rob Clark <robdclark@gmail.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
(cherry picked from commit 6fb7935)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment