Can't figure out how to build a python that uses the .SO file

[mark0978]

All of this is with 2.7.5

We are trying to build on Centos6, the python pyenv builds produces a libpython.a, but not a libpython.so

We have an importable module we build against python using libboostpython. These need to link against a libpython.so

We also need to run under mod_wsgi in apache, so the shims situation is problemmatic. I've set PYTHONHOME to point at the pyenv version of python, but it still goes after 2.6 modules on the main system.

Maybe I'm wrong here, but I am thinking if I could build python to use the .so library, and then link my extension against it, it would work (it does on Debian where I'm NOT using pyenv because I have 2.7 as the base python)

How do you get pyenv to build a python that is based on libpython.so instead of libpython.a?

python is 6MB on Centos, vs 2MB on Debian, I'm assuming that is because libpython.so makes up the other 4MB.

[yyuu]

At first, I don't recommend you to use pyenv always, especially for server applications. The pyenv is mainly designed for development use. If you have problem of pyenv with mod_wsgi, I would recommend you to rebuild dedicated CPython for mod_wsgi without using pyenv.

Second, if you want to create shared library of CPython, you might need to specify --enable-shared as an option of ./configure. Please try to rebuild with env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install -v 2.7.5 to enable shared library. If you have finished building CPython with --enable-shared, you might need to declare proper LD_LIBRARY_PATH to load built shared library.

Third, the binary size issue between CentOS and Debian sounds strange for me. I'd like to check the difference between them.

[tarruda]

@yyuu perhaps pyenv could set LD_LIBRARY_PATH from the shim when invoking a python binary compiled with '--enable-shared'? Setting LD_LIBRARY_PATH explicitly on bashrc has the problem that system applications(ubuntu apps for example) that depend on libpython will be dynamically linked to the wrong file.

It would be nice to build some apps like vim or weechat against a libpython compiled with pyenv

• Update: by default pyenv fails to build python with '--enable-shared' since the installation process will invoke the installed python to complete the installation of pip/setuptools set so pyenv should also set LD_LIBRARY_PATH during installation.

[mark0978]

Here is how we deal with it

env PYTHON_CONFIGURE_OPTS="--enable-shared" LD_LIBRARY_PATH=~/.pyenv/versions/2.7.5/lib/ pyenv install -k 2.7.5

Considering the # of webapps that need mod_wsgi, this really needs to be an option, or maybe even the default.

[yyuu]

The LD_LIBRRY_PATH is for GNU/Linux. Darwin uses DYLD_LIBRARY_PATH as the library search path. We must treat most of platforms properly.

[tarruda]

I found a better way to do it that doesn't depend on pyenv setting LD_LIBRARY_PATH:

PYTHON_CONFIGURE_OPTS='--enable-shared' LDFLAGS="-Wl,-rpath=$HOME/.pyenv/versions/2.7.5/lib" pyenv install 2.7.5

This will add the the python lib directory to the rpath of the python executable. The rpath is searched before LD_LIBRARY_PATH and ld.conf.so, so there's no way pyenv's installed python will try to load the system's libpython which may be of a different version.

[yyuu]

I pushed 43212f6 which will set rpath if --enable-shared was given in PYTHON_CONFIGURE_OPTS. With this fix, just passing --enable-shared will resolve the issue.

$ env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 2.7.5

[mark0978]

Thanks, we just started messing with rpath for some of our SO's yesterday.

[seanfisk]

We just got this working with rpath too! Thanks for being on top of this, @yyuu!

[blueyed]

For reference: when building Vim make sure to only enable either python2 or python3, otherwise Vim's dynamic loading of python will cause Vim to pick up the system's python.
(Or you would have to also add rpath information to Vim or use LD_LIBRARY_PATH).

[pingsoli]

I use fish shell, what should I do ?

env PYTHON_CONFIGURE_OPTS="--enable-shared" LD_LIBRARY_PATH=~/.pyenv/versions/2.7.5/lib/ pyenv install -k 2.7.5 works in bash shell, but not in fish shell.

[pingsoli]

I solved it.

eval (env YTHON_CONFIGURE_OPTS="--enable-shared" pyenv install 3.6.5) works in fish shell.