Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

weird LD_LIBRARY_PATH behaviour #2456

Closed
lucasb-eyer opened this issue Oct 3, 2015 · 9 comments
Closed

weird LD_LIBRARY_PATH behaviour #2456

lucasb-eyer opened this issue Oct 3, 2015 · 9 comments

Comments

@lucasb-eyer
Copy link

@lucasb-eyer lucasb-eyer commented Oct 3, 2015

I don't know if this is a bug, or I'm misunderstanding something here. I have set LD_LIBRARY_PATH as a global export variable, but it's being ignored by ldd (look at the libopenblas.so.0 entry. When I call env LD_LI...=... ldd ... it is correctly taken into account. I'm puzzled.

Welcome to fish, the friendly interactive shell
Type help for instructions on how to use fish
><((("> fish --version
fish, version 2.2.0
><((("> set -l | grep LD_L
><((("> set -g | grep LD_L
LD_LIBRARY_PATH '/home/beyer/inst/lib'  '/opt/cuda/lib64'
history 'set -l | grep LD_L'  'fish --version'  'set -x | grep LD_L'  'set -U '…
><((("> set -U | grep LD_L
><((("> set -x | grep LD_L
LD_LIBRARY_PATH '/home/beyer/inst/lib'  '/opt/cuda/lib64'
><((("> ldd ~/sci-env3/lib/python3.4/site-packages/numpy/linalg/_umath_linalg.cpython-34m.so
    linux-vdso.so.1 (0x00007ffd3f52d000)
    libopenblas.so.0 => not found
    libpython3.4m.so.1.0 => /usr/lib/libpython3.4m.so.1.0 (0x00007fa1ba942000)
    libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0x00007fa1ba617000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007fa1ba319000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fa1ba103000)
    libquadmath.so.0 => /usr/lib/libquadmath.so.0 (0x00007fa1b9ec3000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007fa1b9b1f000)
    libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fa1b9902000)
    libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fa1b96fd000)
    libutil.so.1 => /usr/lib/libutil.so.1 (0x00007fa1b94fa000)
    /usr/lib64/ld-linux-x86-64.so.2 (0x000055eaf1cb7000)
><((("> env LD_LIBRARY_PATH=/home/beyer/inst/lib ldd ~/sci-env3/lib/python3.4/site-packages/numpy/linalg/_umath_linalg.cpython-34m.so
    linux-vdso.so.1 (0x00007ffca6ddb000)
    libopenblas.so.0 => /home/beyer/inst/lib/libopenblas.so.0 (0x00007ff71b3a5000)
    libpython3.4m.so.1.0 => /usr/lib/libpython3.4m.so.1.0 (0x00007ff71aed1000)
    libgfortran.so.3 => /usr/lib/libgfortran.so.3 (0x00007ff71aba6000)
    libm.so.6 => /usr/lib/libm.so.6 (0x00007ff71a8a8000)
    libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007ff71a692000)
    libquadmath.so.0 => /usr/lib/libquadmath.so.0 (0x00007ff71a452000)
    libc.so.6 => /usr/lib/libc.so.6 (0x00007ff71a0ae000)
    libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007ff719e91000)
    libdl.so.2 => /usr/lib/libdl.so.2 (0x00007ff719c8c000)
    libutil.so.1 => /usr/lib/libutil.so.1 (0x00007ff719a89000)
    /usr/lib64/ld-linux-x86-64.so.2 (0x0000557361591000)
><(((">
@faho
Copy link
Member

@faho faho commented Oct 3, 2015

LD_LIBRARY_PATH isn't a list (or rather it is a list but we don't delimit the exported versions with colons) - we only do that for $PATH, $CDPATH and $MANPATH - see the source.

@lucasb-eyer
Copy link
Author

@lucasb-eyer lucasb-eyer commented Oct 3, 2015

That explains the behaviour. But I wonder:

This is deliberately very short - we don't want to add language-specific values like CLASSPATH.

LD_LIBRARY_PATH is not language-specific and I need to set it as often as I need to set PATH. What's the rationale behind including PATH but not LD_LIBRARY_PATH?

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Oct 3, 2015

If LD_LIBRARY_PATH, then we ought to do LD_RUN_PATH, and then DYLD_LIBRARY_PATH too, in which case we ought to do DYLD_FRAMEWORK_PATH, and so on. It just keeps going.

I'm very open to ideas here but I think an ever-expanding whitelist is not the solution.

@lucasb-eyer
Copy link
Author

@lucasb-eyer lucasb-eyer commented Oct 3, 2015

The second thing I was wondering was

both incoming from the environment and outgoing back to it

why not always convert "outgoing" arrays to colon-delimited, instead of 0x1e? But I imagine you guys already thought of that and have good reasons against it.

Besides that, I have no other ideas myself, unfortunately. None of this is satisfying.

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Oct 3, 2015

The issue with doing just outgoing arrays is recursive invocations of fish. It would be weird if a variable went from an array to a string just by reinvoking fish.

@lucasb-eyer
Copy link
Author

@lucasb-eyer lucasb-eyer commented Oct 3, 2015

I guessed so. AFAICT, though, it's really only fish. Anything else (I know of) is being confused by 0x1e, even though it's the "more correct" separator. But I can see how special-casing "outgoing to fish" vs. "outgoing to the rest of the world" is also far from ideal.

I'm out of ideas and will just work around this specific instance of the issue through a set -xg LD_LIBRARY_PATH (printf '%s\n' $LD_LIBRARY_PATH | paste -sd:).

I'm closing this issue, but if you want to keep it open as a reminder, feel free to reopen.

@lucasb-eyer lucasb-eyer closed this Oct 3, 2015
@zanchey
Copy link
Member

@zanchey zanchey commented Oct 9, 2015

At least fish can export arrays to child shells! It doesn't work at all in bash or zsh.

In bash or zsh, you always have to hand-mangle LD_LIBRARY_PATH, so I don't think this is actually a step backwards.

@cben
Copy link
Contributor

@cben cben commented Dec 21, 2015

There is one natural solution that covers most common uses without requiring future changes:
Whitelist = everything ending in PATH.

Sure, there are other colon-delimited env vars e.g. XDG_DATA_DIRS but *PATH is most common and — importantly — a very simple rule.
It's actually simpler to remember and leads to more consistent code than current rule. Why should PATH-manipulation and CLASSPATH-manipulation code look different?

@ridiculousfish you mentioned *PATH somewhere as "bad idea". Could you please elaborate why?

I'd even (half seriously) question why the whitelist is not empty!
Of course dropping $PATH would be backward-incompatible for a LOT of users. But you were willing to break PYTHONPATH etc...
Or at least why it's not just PATH?


Another way to look at it is that handling :-delimited strings is currently insufficiently easy.
Writing single code that handles :-delimited strings and arrays equally is even harder (most notably, "$PATH" being space-separated instead of the original value is annoying).
If we had something like #1656 (comment), we'd be able to write concise stuff like for d in $XDG_DATA_DIRS@: or env PATH=/foo:{$PATH@:} cmd and wouldn't seriously need automagic colon-splitting.

@krader1961
Copy link
Contributor

@krader1961 krader1961 commented Dec 21, 2015

Can we please continue this discussion in issue #436? There's also issue #2090 which talks about setting MANPATH. And issue #2596. And probably several more. Argh! :-)

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 18, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants