Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python gui.so links to wrong QGIS libs when more than one version of QGIS is installed #11608

Closed
qgib opened this issue Feb 17, 2009 · 8 comments

Comments

@qgib
Copy link
Contributor

qgib commented Feb 17, 2009

Author Name: Tim Sutton (Tim Sutton)
Original Redmine Issue: 1548

Redmine category:build/install
Assignee: Martin Dobias


Scenario:

I install my development builds to /home/timlinux/apps/

Since we adopted versioned lib names (e.g. libqgis_core.so.1.1) it should be possible to have more than one version of QGIS coexsisting in the same CMAKE_INSTALL_PREFIX. This does indeed work and all libs and apps link to the correct libs except the python generated libs under /share/qgis/python/qgis/core.so and /share/qgis/python/qgis/gui.so, as the listings that follow show:

 ldd gui.so | grep qgis
	libqgis_core.so.1.1 => /home/timlinux/apps/lib/libqgis_core.so.1.1 (0x00007f598b893000)
	libqgis_gui.so.1.0 => /home/timlinux/apps/lib/libqgis_gui.so.1.0 (0x00007f598aa47000)
	libqgis_core.so.1.0 => /home/timlinux/apps/lib/libqgis_core.so.1.0 (0x00007f5983624000)

and

[qgis] ldd core.so | grep qgis
	libqgis_core.so.1.0 => /home/timlinux/apps/lib/libqgis_core.so.1.0 (0x00007fbce8b98000)

You can see some strange things are going on here.

  • Firstly they should all be linking to 1.1 libs since this build is a 1.1 build.
  • Secondly, instead of being called core.so, these libs should be core.so.1.0 and core.so.1.1 libs so that the appropriate python libs are loaded based on the version of QGIS running (1.0 or 1.1)
  • thirdly gui.so seems to be linked to both 1.0 and 1.1 of libqgis_core

It will be great if you can resolve this so that the core and gui libs are versioned and linking to the appropriate version of qgis core and gui libs.

Many thanks

Tim

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Feb 18, 2009

Author Name: Martin Dobias (@wonder-sk)


Hi Tim,

installing two version of qgis in one install prefix doesn't make much sense to me. The "sonames" we're using for libraries are meant to help the compatibility, e.g. if there's an app requiring library with soname 1.1 and you have the library in version 1.3.2 with soname 1.1, the linker knows this will work.

While this allows different versions of libraries to coexist in one path, we must think also of other parts of QGIS: binary "qgis" is not versioned, thus overwritten with a newer install, plugins are not versioned (overwritten too), resources can exist also only in one version... The same is for python bindings.

But imagine that it would be possible to have pyqgis modules core.so.1.0 and core.so.1.1 in the same path. When you open python console and type: import qgis.core - how should the interpreter know which module are you asking for? To my knowledge, python fails to load the module if you try to rename it from core.so to core.so.X.Y.

The problem with the linking (pyqgis modules linking to both 1.0 and 1.1 libs) is surely caused by the mixture of two different builds in your build directory. Doing a clean compilation will fix the problem. I don't know how to force the linker to choose library with right soname. As far as I understand it, the linker checks for soname of the shared object you're linking against and then dynamic linker loads the library with correct soname.

I hope I have convinced you that keeping two different versions of QGIS in one path is a bad idea and this ticket is invalid :-)

Martin

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Feb 18, 2009

Author Name: Tim Sutton (Tim Sutton)


Hi Martin

Reading back to my original post, I see I didnt explain something well. I am not trying to run two concurrent versions from the same location (as you say some things obviously wont work in this context like the fact that the QGIS binary is being overwritten). I am however trying to install two versions to the same location (using only the last installed at any time though). Let me explain better:

I have two QGIS source directories, one for trunk and one for stable. Each one has its own build dir. And both install to the same prefix. When I want to work with QGIS 1.0.x I run a make install in the stable source dir build directory and then run QGIS. Same thing for QGIS 1.1. I have followed this practice for some time without issues. Recently things have changed - I guess since we started 1.1 lib name versioning and the possibility arrived for linking to the two libs.

"
But imagine that it would be possible to have pyqgis modules core.so.1.0 and core.so.1.1 in the same path. When you open python console and type: import qgis.core - how should the interpreter know which module are you asking for?"

My expectation was that if core.so is linked to only one qgis_core lib, the interpreter should use the one linked to.

"To my knowledge, python fails to load the module if you try to rename it from core.so to core.so.X.Y. "

Ok just having gui.so and core.so link to only one lib would be a satisfactory solution here.

"The problem with the linking (pyqgis modules linking to both 1.0 and 1.1 libs) is surely caused by the mixture of two different builds in your build directory. Doing a clean compilation will fix the problem. I don't know how to force the linker to choose library with right soname. As far as I understand it, the linker checks for soname of the shared object you're linking against and then dynamic linker loads the library with correct soname. "

This is the heart of the matter. Note that I am working with a clean build directory i.e. it contains no old object files from pre 1.1. For some reason in recent weeks something has changed such that overwriting installs between two different QGIS version causes odd linking in the python generated libs.

I appreciate that my useage is an edge case however as I mentioned above, my approach used to work fine until the last 3 or 4 weeks (guessing on timeframe here). I just deleted my build dir in trunk sources and built again into my standard prefix to double / triple check and I still get the same cross linking side effect:

[qgis] ldd gui.so  | grep qgis
	libqgis_core.so.1.1 => /home/timlinux/apps/lib/libqgis_core.so.1.1 (0x00007feb02f1a000)
	libqgis_gui.so.1.0 => /home/timlinux/apps/lib/libqgis_gui.so.1.0 (0x00007feb020ce000)
	libqgis_core.so.1.0 => /home/timlinux/apps/lib/libqgis_core.so.1.0 (0x00007feafacaa000)

As such I am not ready to close this bug as invalid at this point as there is definately something odd going on...

Best regards

Tim

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Feb 18, 2009

Author Name: Jürgen Fischer (@jef-n)


Does the following patch help?

Index: CMakeLists.txt
===================================================================
--- CMakeLists.txt	(revision 10172)
+++ CMakeLists.txt	(working copy)
@@ -319,6 +319,7 @@
 INSTALL(FILES ${CMAKE_BINARY_DIR}/qgsconfig.h DESTINATION ${QGIS_INCLUDE_DIR})
 
 INCLUDE_DIRECTORIES(${CMAKE_BINARY_DIR})
+LINK_LIBRARIES(${CMAKE_BINARY_DIR}/src/core ${CMAKE_BINARY_DIR}/src/gui)
 
 #############################################################
 # create qgssvnversion.h

I applied this to both Version_1.0 and trunk and your problem doesn't occur, but I didn't try to reproduce the problem without it.

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Feb 18, 2009

Author Name: Martin Dobias (@wonder-sk)


Replying to [comment:2 timlinux]:

I have two QGIS source directories, one for trunk and one for stable. Each one has its own build dir. And both install to the same prefix. When I want to work with QGIS 1.0.x I run a make install in the stable source dir build directory and then run QGIS. Same thing for QGIS 1.1. I have followed this practice for some time without issues. Recently things have changed - I guess since we started 1.1 lib name versioning and the possibility arrived for linking to the two libs.

Ah ok, now I go it :-)

"
But imagine that it would be possible to have pyqgis modules core.so.1.0 and core.so.1.1 in the same path. When you open python console and type: import qgis.core - how should the interpreter know which module are you asking for?"

My expectation was that if core.so is linked to only one qgis_core lib, the interpreter should use the one linked to.

That's correct expectation.

I appreciate that my useage is an edge case however as I mentioned above, my approach used to work fine until the last 3 or 4 weeks (guessing on timeframe here). I just deleted my build dir in trunk sources and built again into my standard prefix to double / triple check and I still get the same cross linking side effect:

> [qgis] ldd gui.so  | grep qgis
> 	libqgis_core.so.1.1 => /home/timlinux/apps/lib/libqgis_core.so.1.1 (0x00007feb02f1a000)
> 	libqgis_gui.so.1.0 => /home/timlinux/apps/lib/libqgis_gui.so.1.0 (0x00007feb020ce000)
> 	libqgis_core.so.1.0 => /home/timlinux/apps/lib/libqgis_core.so.1.0 (0x00007feafacaa000)

Ok so the linker picks for some reason the wrong libs. I don't have an advice for you what exactly to do. My ideas are: go to (build_dir)/python/core and (build_dir)/python/gui and check the generated makefiles - especially the LIBS variable to see what paths are used when searching for libraries.

If you won't be able to find the reason, we can take a look at it on the hackfeast.

Martin

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Feb 19, 2009

Author Name: Tim Sutton (Tim Sutton)


Jefs patch above (though using LINK_DIRECTORIES instead of LINK_LIBRARIES) partially fixes the issue in that the core and gui libs are no longer double linked to two different versions of QGISCORE/QGISGUI. However on starting QGIS I still get an error

Couldn't load [[PyQGIS]].
Python support will be disabled.

Traceback (most recent call last):
  File "", line 1, in 
[[ImportError]]: No module named qgis.core

Python version:
2.5.2 (c5cd2930 (SVN r253):60911, Oct  5 2008, 19:49:54) 
[GCC 4.3.2]

Python path:
['/python', '/home/timlinux/.qgis//python/plugins', '/python/plugins', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/Numeric', '/usr/lib/python2.5/site-packages/PIL', '/usr/lib/python2.5/site-packages/gst-0.10', '/var/lib/python-support/python2.5', '/usr/lib/python2.5/site-packages/gtk-2.0', '/var/lib/python-support/python2.5/gtk-2.0', '/usr/lib/python2.5/site-packages/wx-2.6-gtk2-unicode']

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Feb 23, 2009

Author Name: Martin Dobias (@wonder-sk)


This will be caused by C++ plugins for older version of QGIS. I still don't know why exactly this happens, but if you leave in (prefix)/lib/qgis some plugins for older version of QGIS (i.e. they link to some other version of qgis_core and qgis_gui libs), [[QgsApplication]] returns empty strings instead of returning correct paths. You can see from you error message that the python path is set to directories like /python or /python/plugins which simply don't include the prefix.

You can work this around by checking the plugins in the install prefix and delete those for the older version (e.g. gridmaker?).

Martin

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Jul 15, 2009

Author Name: Paolo Cavallini (@pcav)


I reclass this, as it does not seem critical. Please change it again if I was wrong

@qgib

This comment has been minimized.

Copy link
Contributor Author

qgib commented Nov 7, 2009

Author Name: Paolo Cavallini (@pcav)


  • resolution was changed from to invalid
  • status_id was changed from Open to Closed
@qgib qgib added this to the Version 1.4.0 milestone May 24, 2019
@qgib qgib closed this May 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.