Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python lib suffix detection flawed (kind of) #17

Closed
TeHMoroS opened this issue Jun 30, 2014 · 7 comments
Closed

Python lib suffix detection flawed (kind of) #17

TeHMoroS opened this issue Jun 30, 2014 · 7 comments

Comments

@TeHMoroS
Copy link

Hi.

This won't be very precise as my knowlegde of automake tools quite limited, but I found a little bug. It's for the current version. The python.m4 macro has some code that evaluates the libdirsuffix variable, later used in the PYTHON_LIB_LOC variable and so on. The libdrisuffix is hardcoded with the "/i386-linux-gnu/".

Here's the problem: the evaluation of the libdirsuffix variable passes without making changes to the initial value. This results in a "/i386-linux-gnu/" suffix... on a 64-bit environment... where the directory doesn't exist. ;)

This results in an error when trying to load a Python extension (RabbitVCS Caja extension in my case):

(caja:1258): Caja-Python-WARNING **: g_module_open libpython failed: /usr/lib/i386-linux-gnu//libpython2.7.so.1.0: (translated from polish) cannot open shared object file: No such file or directory
ImportError: could not import gobject (error was: '/usr/lib/python2.7/site-packages/gi/_gi.so: undefined symbol: _Py_ZeroStruct')

The workaround for me was to change the hardcoded "/i386-linux-gnu/" to just "/", but it seems that the path evalution code didn't do anything on my environment. I'm using ArchLinux 64-bit. The basic directory structure for the lib directories is:

  • /usr/lib - 64-bit libraries
  • /usr/lib32 - 32-bit libraries
  • /usr/lib64 - symlink to /usr/lib

I didn't have too much time to dig into the evaluation code (at work right now and really needed that SVN integration), but if I find anything else, I'll post it here. If there's anything else you would need to know, I'll be happy to add it.

@TeHMoroS
Copy link
Author

Ok, I've got some clues on what might be the problem here on ArchLinux. First things first. The compiler search path output is (libraries part in bold):

install: /usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/ programs: =/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/bin/ libraries: =/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/../lib/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../lib/:/lib/x86_64-unknown-linux-gnu/4.9.0/:/lib/../lib/:/usr/lib/x86_64-unknown-linux-gnu/4.9.0/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../../x86_64-unknown-linux-gnu/lib/:/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.0/../../../:/lib/:/usr/lib/

The python.m4 macro is based on the assupmtion that if there's a /lib64 suffix in the libraries search path, then we've got a 64-bit ABI.
There's also a general assumption that 32-bit libraries go under $prefix/lib and 64-bit ones under $prefix/lib64. Why? It's only natural, that native libaries go under /lib and non-native under /lib(emulated arch bit number here).

Arch uses that schema:

  • /usr/lib is where 64-bit libraries go and /usr/lib32 is where 32-bit libraries go for a 64-bit environment;
  • /usr/lib is where libraries go for a 32-bit environment;

The compiler never outputs anything like "/lib32" or "/lib64", only "/lib", thus the detection code will never find anything and fall back to the first value of $libdirsuffix (which is kind of "ubuntish" anyway...).

@infirit
Copy link
Contributor

infirit commented Jul 11, 2014

Python provides a pkgconfig file which we can use instead of the custom macro. I'll have a look.

@infirit
Copy link
Contributor

infirit commented Jul 11, 2014

Can you try the below and see if it fixes the problem?

 diff --git a/configure.ac b/configure.ac
index f4059e3..66bc531 100644
--- a/configure.ac
+++ b/configure.ac
@@ -30,8 +30,12 @@ GTK_DOC_CHECK(1.9)
 dnl **************************************************
 dnl * Check for Python
 dnl **************************************************
-AM_CHECK_PYTHON_HEADERS(,[AC_MSG_ERROR([could not find Python headers])])
-AM_CHECK_PYTHON_LIBS(,[AC_MSG_ERROR([could not find Python lib])])
+AM_PATH_PYTHON([2.6])
+PKG_CHECK_MODULES([PYTHON], [python-${PYTHON_VERSION}])
+PYTHON_LIB_LOC="pkg-config --variable=libdir'"
+AC_SUBST(PYTHON_LIBS)
+AC_SUBST(PYTHON_CFLAGS)
+AC_SUBST(PYTHON_LIB_LOC)

 if test "`pkg-config --variable=datadir pygobject-3.0`" != "" ; then
     PYGOBJECT_VERSION=pygobject-3.0
diff --git a/src/Makefile.am b/src/Makefile.am
index e3d7c3f..b9cbd55 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -16,12 +16,13 @@ libcaja_python_la_CPPFLAGS = \
    -DLIBDIR=\"$(libdir)\" \
    -DPYTHON_VERSION=\"$(PYTHON_VERSION)\" \
    -DPY_LIB_LOC="\"$(PYTHON_LIB_LOC)\"" \
-   $(PYTHON_INCLUDES) \
+   $(CAJA_PYTHON_CFLAGS) \
    $(AM_CPPFLAGS)

 libcaja_python_la_CFLAGS = \
+   $(PYTHON_CFLAGS)
    $(CAJA_PYTHON_CFLAGS) \
    $(AM_CFLAGS)

 libcaja_python_la_LDFLAGS = -module -avoid-version
-libcaja_python_la_LIBADD  = $(CAJA_PYTHON_LIBS) $(PYTHON_LIBS)
+libcaja_python_la_LIBADD  = $(PYTHON_LIBS) $(CAJA_PYTHON_LIBS) 

@TeHMoroS
Copy link
Author

The compilation failed for this patch:

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -DG_LOG_DOMAIN=\"Caja-Python\" -DDATADIR=\"/usr/share\" -DLIBDIR=\"/usr/lib\" -DPYTHON_VERSION=\"2.7\" -DPY_LIB_LOC= -pthread -I/usr/include/pygobject-3.0 -I/usr/lib/libffi-3.1/include -I/usr/include/caja -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/libdrm -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -D_FORTIFY_SOURCE=2 -pthread -I/usr/include/pygobject-3.0 -I/usr/lib/libffi-3.1/include -I/usr/include/caja -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/libdrm -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 -I/usr/include/pango-1.0 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/harfbuzz -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -MT libcaja_python_la-caja-python.lo -MD -MP -MF .deps/libcaja_python_la-caja-python.Tpo -c caja-python.c  -fPIC -DPIC -o .libs/libcaja_python_la-caja-python.o
caja-python.c:24:20: fatal error: Python.h: No such file or directory
 #include <Python.h>
                    ^
compilation terminated.
Makefile:473: recipe for target 'libcaja_python_la-caja-python.lo' failed

If it helps anything, Arch is building python-caja based on this source package: http://pub.mate-desktop.org/releases/1.8/python-caja-1.8.0.tar.xz

@infirit
Copy link
Contributor

infirit commented Jul 11, 2014

I suck 😢 and messed up the patch. New version here https://github.com/infirit/python-caja/commit/6ed4ebcf1af8eaead423de1aefe2642f2d75a979.diff.

PS: Do make sure to run autoreconf -fi to regen the build system before running configure 😄

@TeHMoroS
Copy link
Author

It works. 😄 Thanks infirit!
It went smoothly this time. The resulting binary also seems good (for example, the RabbitVCS extension works fine), internal references point to /usr/lib/libpython2.7.so.1.0 (which exists) instead of /usr/lib/i386-linux-gnu//libpython2.7.so.1.0 and no more log messages for gobject import errors. 😄

@infirit
Copy link
Contributor

infirit commented Jul 11, 2014

Excellent, pushed as 6ed4ebc.

@infirit infirit closed this as completed Jul 11, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants