Add support for setting LD_LIBRARY_PATH in bash #326

Closed
wants to merge 1 commit into
from

Projects

None yet

9 participants

@tomspur
tomspur commented Sep 3, 2012 edited

When activating the virtualenv, prepend $VIRTUAL_ENV/lib to LD_LIBRARY_PATH

Tested with: GNU bash, version 4.2.28(1)-release

@tomspur tomspur Add support for setting LD_LIBRARY_PATH in bash
Signed-off-by: Thomas Spura <thomas.spura@gmail.com>
d3e2243
@travisbot

This pull request passes (merged d3e2243 into 9f0d0c9).

@carljm
Contributor
carljm commented Sep 3, 2012

I don't see any problem with this. @jezdez, @pnasrat, @qwcode? Only thing that strikes me is it'd be a bit odd to add support for this in activate.sh but not the equivalent scripts for other shells.

@tomspur
tomspur commented Sep 3, 2012

@carljm I can try to implement it in the other shell scripts too. But they would be untested, as I don't use them... That's why I preferred to leave them out for now.

Are there only bash tests available? When travis also tests the other shells, we could also add a test for the library path and be on the save side for all kinds of shells?

@carljm
Contributor
carljm commented Sep 3, 2012

Yeah, that's the problem with maintaining all these activation scripts, nobody uses all of the shells so getting a change into the activation script becomes difficult.

I'd like to hear other maintainers' opinions on this.

Virtualenv's tests are far from complete, any addition to test coverage would be great.

@pnasrat
Contributor
pnasrat commented Sep 3, 2012

I'd like to hear what is your actual use case for this. What specifically are you using that requires this to be set? For python native extensions we should currently do the correct thing - are you building native deps directly into virtualenv locations?

My personal view is that I don't think we should be making virtualenv a chroot/native OS container replacement. This feels like a step along towards that, but without putting the design/testing in place to make it robust. Adding this in activate.sh, without making it part of the interpreter when we run means - eg if you're trying to use ctypes to load a native lib with dep chain built into the virtualenv you may break if activate.sh is not called.

@tomspur
tomspur commented Sep 3, 2012

A few days ago, I started to write a little script, which installs e.g. all major dependencies of ipython into a virtualenv. This works fine with ./configure --prefix=$VIRTUAL_ENV, but LD_LIBRARY_PATH needs to be set manually right now (for instance it's still quite tricky to get yt working).
If it's fine with you, I'll submit more PR into this direction, so the environment works also for autotools based repositories and have a look at other ones then.

chroot is not available as an user and this was my main motivation for using virtualenv as a replacement for a (limited) isolated environment in user space...

@MatthewTurk

This would definitely be very useful for us. Right now we (yt developers) ship a version of the activate script so that we can have semi-isolation for the packages contained therein with LD_LIBRARY_PATH. Our particular use case is that a number of our tools depend on C libraries which we build (independent of the -- often shared-user -- base system) and test specific to a given revision. Having LD_LIBRARY_PATH support would mean we could use the virtualenv system from top-to-bottom, and would help us to package and distribute yt, as @tomspur has noted above.

@pnasrat
Contributor
pnasrat commented Nov 15, 2012

Larger feature will look at after next point release

@bak1an
bak1an commented Jan 11, 2013

+1 for this

Also maybe we can also do something like that:

CPATH=$VIRTUAL_ENV/include:$CPATH
LIBRARY_PATH=$VIRTUAL_ENV/lib:$LIBRARY_PATH

I used this to install pylibmc on old centos.
I've compiled and installed libmemcached with --prefix=$VIRTUAL_ENV and with above enviroment i was able to just 'pip install pylibmc' and get everything compiled/installed.

@nextdoor-engineer

+1 from me, too. Except that the request doesn't ask for enough. virtualenv should provide options to add to PYTHONPATH and to LD_LIBRARY_PATH to support packages that are installed by a method other than virtualenv (by apt-get, in my case.) Here's what I do now. There must be a cleaner and more portable way.

POSTGRES_PKGS = postgresql-9.1 postgresql-9.1-client notfound
$(REQUIREMENTS_FILE)_install:
        # Install the packages from one requirements file to the virtual environment.
        set -e; set -x; for p in $(POSTGRES_PKGS); do \
            if test $$p = notfound; then \
                echo "Did not find an installed postgresql. Exiting."; \
                dpkg-query -l "postgresql*" \
                exit 1; \
            elif q=`dpkg-query -L $$p | grep -E ".*postgres.*bin$$"`; then \
                echo 'PATH='$$q':$$PATH' >> $(VENV)/bin/activate; \
                echo 'LD_LIBRARY_PATH='`dirname $$q`'/lib:$$LD_LIBRARY_PATH' >> \
                    $(VENV)/bin/activate; \
                echo 'export LD_LIBRARY_PATH' >> $(VENV)/bin/activate; \
                break; \
            fi; \
        done
@serge-sans-paille

Resurrecting the thread, with an additional use case:

  1. create a virtual env
  2. install a third party shared library and its python wrapper using <my_env> as a prefix
  3. import the wrapper that itself loads the shared library

The problem is that the shared library gets install into <my_env>/lib (which is a normal behavior) but virtualenv's activate does not update LD_LIBRARY_PATH.

@BrownTruck

Hello!

As part of an effort to ease the contribution process and adopt a more standard workflow pip has switched to doing development on the master branch. However, this Pull Request was made against the develop branch so it will need to be resubmitted against master. Unfortunately, this pull request does not cleanly merge against the current master branch.

If you do nothing, this Pull Request will be automatically closed by @BrownTruck since it cannot be merged.

If this pull request is still valid, please rebase it against master (or merge master into it) and resubmit it against the master branch, closing and referencing the original Pull Request.

If you choose to rebase/merge and resubmit this Pull Request, here is an example message that you can copy and paste:

When activating the virtualenv, prepend $VIRTUAL_ENV/lib to  LD_LIBRARY_PATH

Tested with: GNU bash, version 4.2.28(1)-release

---

*This was migrated from pypa/virtualenv#326 to reparent it to the ``master`` branch. Please see original pull request for any previous discussion.*
@BrownTruck

This Pull Request was closed because it cannot be automatically reparented to the master branch and it appears to have bit rotted.

Please feel free to re-open it or re-submit it if it is still valid and you have rebased it onto master or merged master into it.

@BrownTruck BrownTruck closed this May 26, 2016
@tomspur tomspur referenced this pull request May 27, 2016
Open

Add support for setting LD_LIBRARY_PATH in bash #923

0 of 3 tasks complete
@tomspur
tomspur commented May 27, 2016

I just opened #923 as a follow-up as I was not allowed to reopen this pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment