Binaries may not expect shared libraries in /lib{,64}

[weidendo]

All shared libraries get copied into /lib or /lib64 in the "container".
This does not work with binaries from Ubuntu, as they expect shared libraries (have rpath set) in "/lib/x86_64-linux-gnu/", even for "/bin/sh". Explicitly setting LD_LIBRARY_PATH in src/sexec.c with setenv(2) helps.

[gmkurtzer]

Ahh, good find! I think I have a very portable way of fixing this and will implement as soon as possible and let you know for testing.

Thanks!

[gmkurtzer]

I updated the ldd dependency resolver to use the default library paths and use LD_LIBRARY_PATH to update the search path. Will you recreate your SAPPs and let me know what you find?

Thank you!

[weidendo]

Looks good!

[weidendo]

I closed to early. When an application itself requires LD_LIBRARY_PATH set, the libraries are fetched into the sapp, but they are not found on execution. Am I missing something?

[gmkurtzer]

Ahhh, that is a good catch. Some variables should be caught and replicated into the SAPP. I'll think about how to do it and implement something when I get to the computer. Let me know if you think of a fix too!

Thanks!!

Sent from my iPhone

On Nov 27, 2015, at 10:33 AM, weidendo notifications@github.com wrote:

I closed to early. When an application itself requires LD_LIBRARY_PATH set, the libraries are fetched into the sapp, but they are not found on execution. Am I missing something?

—
Reply to this email directly or view it on GitHub.

[gmkurtzer]

I just committed a method which will find all libraries in the SAPP and include them in the LD_LIBRARY_PATH. Can you check to see if that is working now? Thanks!

[weidendo]

Hmm, no.
I compiled something with ICC, which needs
/opt/intel/composer_xe_2015.5.223/compiler/lib/intel64
to be in LD_LIBRARY_PATH.

...> singularity build heat.spec
Building: heat
Checking paths...
Checking writable...
Evaluating: %packages
Creating template configs
/etc/nsswitch.conf
/etc/resolv.conf
/etc/hosts
Evaluating: hard requirements
Installing file: /bin/sh
Evaluating: %exec
Installing file: /home/weidendo/SW/heat/heat
Evaluating: %files
Invoking dependency check
Installing file: /lib/x86_64-linux-gnu/libm.so.6
Installing file: /opt/intel/composer_xe_2015.5.223/compiler/lib/intel64/libiomp5.so
Installing file: /lib/x86_64-linux-gnu/libgcc_s.so.1
Installing file: /lib/x86_64-linux-gnu/libpthread.so.0
Installing file: /lib/x86_64-linux-gnu/libc.so.6
Installing file: /lib/x86_64-linux-gnu/libdl.so.2
Installing file: /lib64/ld-linux-x86-64.so.2
Including SAPPFILE
Calculating container size
Calculating checksums
Writing metadata
Building primary singularity exec driver
Building singularity test runscript
Building singularity main runscript
Running %test...
Cleaning up unpackaged files
Creating output SAPP container
WROTE: heat.sapp
weidendo@lapbode161:~/SW/heat (master)> ./heat.sapp
/home/weidendo/SW/heat/heat: error while loading shared libraries: libiomp5.so: cannot open shared object file: No such file or directory

What is your way to debug this?

[weidendo]

Looking into ~/.sapp-cache/heat/c/singularity:
...
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu
/opt/intel/composer_xe_2015.5.223/compiler/lib/intel64"
...

Newline instead of colon?

[gmkurtzer]

Woops... Sorry about that! I just committed a fix that will work... Will test as soon as I get into work.

[gmkurtzer]

The fix seems to be working for me, but I don't have a use-case on this system with multiple libdirs. Can you confirm the latest commits are working better?

Thanks!

[weidendo]

Works fine here.

[gmkurtzer]

Excellent, thanks!