add support for handling dependencies via RPATH #651

boegel · 2013-06-02T20:01:11Z

Using RPATH as an alternative to modules for handling dependencies should be supported by the EasyBuild framework.

The text was updated successfully, but these errors were encountered:

fgeorgatos · 2013-06-11T22:34:30Z

...and this might just solve a bug that we are noticing in a particular test case with the iomkl stack:

we think that openmpi's orted cannot find its pieces (under OAR and similar schedulers), unless it is assisted by the system in relation to LD_LIBRARY_PATH.
You may never notice this if you are with PBS because it inherits the environment form the submitting process, using its own mechanism (SGE provides for -V option for the same reason).

The solution to this could be perhaps rpath or a static compile (great as a firework for an upcoming hackathon).

boegel · 2013-06-12T07:12:08Z

I think gradually moving to static or RPATH builds regardless of whether there are any observed problems is good in an HPC setting...

fgeorgatos · 2013-11-17T20:48:52Z

Having the option to build the world with RPATH is good; making it compulsory though, I think not really:
http://www.open-mpi.org/faq/?category=all#why-no-rpath

(in fact, it defeats the great feature of recent OpenMPI stacks of being ABI-compatible, which allows important freedom for some experimentation - module swap is your friend)

addendum: ditto for static, ref. http://www.open-mpi.org/faq/?category=all#static-mpi-apps , 101 & 102

boegel · 2015-08-12T15:41:22Z

@pforai: do you have more details on the ld wrapper that CSC.fi uses for making RPATH linking easy?

wpoely86 · 2015-12-16T09:31:09Z

I had another look at this for easybuilders/easybuild-easyconfigs#2228

We should just use a wrapper for ld with contents:

#!/bin/bash

ld @$EBLINKEROPTIONS $@

Where $EBLINKEROPTIONS must point to a file with all the linker options. This file must exists but can be empty.

pforai · 2015-12-16T09:33:18Z

@wpoely86 @boegel Torben Rasmussen's from Sweden have such a script, we can ask them how it looks like.

boegel · 2015-12-16T09:37:40Z

@pforai: up for firing them a mail on it? Not sure if he's on GitHub.

pforai · 2015-12-16T09:41:12Z

@boegel just dispatched, you in CC.

rjeschmi · 2016-01-29T19:13:59Z

Is this where we are working on this?

In any case. GCC calls collect2 rather than ld and seems to be pretty good at finding the real ld.

Not sure what I should override. Maybe I can set an environment variable to force it to use my special ld?

rjeschmi · 2016-01-29T19:18:15Z

access("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/collect2", X_OK) = 0
[pid  4142] execve("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/collect2", ["/software/easybuild/software/GCC"..., "-plugin", "/software/easybuild/software/GCC"..., "-plugin-opt=/software/easybuild/"..., "-plugin-opt=-fresolution=/tmp/cc"..., "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "-plugin-opt=-pass-through=-lpthr"..., "-plugin-opt=-pass-through=-lc", "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "--eh-frame-hdr", "-m", "elf_x86_64", "-dynamic-linker", "/lib64/ld-linux-x86-64.so.2", ...], [/* 104 vars */] <unfinished ...>
[pid  4142] <... execve resumed> )      = 0
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/real-ld", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/real-ld", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/real-ld", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/collect-ld", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/collect-ld", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/collect-ld", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/ld.gold", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/ld.gold", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/ld.gold", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/gnm", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/gnm", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/gnm", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/nm", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/nm", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/nm", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/gstrip", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/gstrip", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/gstrip", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/strip", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/strip", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/strip", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/gcc", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/gcc", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4142] stat("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/gcc", 0x7ffcc99185b0) = -1 ENOENT (No such file or directory)
[pid  4143] execve("/bin/ld.gold", ["/bin/ld.gold", "-plugin", "/software/easybuild/software/GCC"..., "-plugin-opt=/software/easybuild/"..., "-plugin-opt=-fresolution=/tmp/cc"..., "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "-plugin-opt=-pass-through=-lpthr"..., "-plugin-opt=-pass-through=-lc", "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "--eh-frame-hdr", "-m", "elf_x86_64", "-dynamic-linker", "/lib64/ld-linux-x86-64.so.2", ...], [/* 104 vars */] <unfinished ...>
[pid  4143] <... execve resumed> )      = 0
[pid  4143] open("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/liblto_plugin.so", O_RDONLY|O_CLOEXEC) = 3
[easybuild@de0bb4c6145e samtools-1.2]$ echo $PATH
/tmp/eb-A0ozfo/eb-ldwrapper-sTMN3t:/export/easybuild/software/SAMtools/1.2-foss-2015a/bin:/export/easybuild/software/ncurses/5.9-foss-2015a/bin:/tmp/eb-A0ozfo/eb-ldwrapper-sTMN3t:/software/easybuild/software/FFTW/3.3.4-gompi-2015a/bin:/software/easybuild/software/OpenBLAS/0.2.13-GCC-4.9.2-LAPACK-3.5.0/bin:/software/easybuild/software/OpenMPI/1.8.4-GCC-4.9.2/bin:/software/easybuild/software/hwloc/1.10.0-GCC-4.9.2/bin:/software/easybuild/software/numactl/2.0.10-GCC-4.9.2/bin:/software/easybuild/software/GCC/4.9.2/bin:/software/easybuild/software/FPM/1.3.3-Ruby-2.1.6/bin:/software/easybuild/software/Ruby/2.1.6/bin:/software/easybuild-develop/easybuild-framework:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/easybuild/.local/bin:/home/easybuild/bin
[easybuild@de0bb4c6145e samtools-1.2]$ which ld
/tmp/eb-A0ozfo/eb-ldwrapper-sTMN3t/ld

rjeschmi · 2016-01-29T19:21:36Z

seems like it is ok if I get ld.gold in the path.

rjeschmi · 2016-01-29T19:28:28Z

[easybuild@de0bb4c6145e samtools-1.2]$ make
gcc -pthread  -o samtools bam_index.o bam_plcmd.o sam_view.o bam_cat.o bam_md.o bam_reheader.o bam_sort.o bedidx.o kprobaln.o bam_rmdup.o bam_rmdupse.o bam_mate.o bam_stat.o bam_color.o bamtk.o bam2bcf.o bam2bcf_indel.o errmod.o sample.o cut_target.o phase.o bam2depth.o padding.o bedcov.o bamshuf.o faidx.o stats.o stats_isize.o bam_flags.o bam_split.o bam_tview.o bam_tview_curses.o bam_tview_html.o bam_lpileup.o libbam.a htslib-1.2.1/libhts.a  -L/export/easybuild/software/ncurses/5.9-foss-2015a/lib -lcurses  -lm -L/export/easybuild/software/zlib/1.2.8-foss-2015a/lib -lz
INFO: linking with rpath
INFO: linking with rpath and NSC symbols
INFO: RPATH : -rpath=/export/easybuild/software/SAMtools/1.2-foss-2015a/lib -rpath=/export/easybuild/software/ncurses/5.9-foss-2015a/lib -rpath=/export/easybuild/software/zlib/1.2.8-foss-2015a/lib -rpath=/software/easybuild/software/FFTW/3.3.4-gompi-2015a/lib -rpath=/software/easybuild/software/GCC/4.9.2/lib -rpath=/software/easybuild/software/GCC/4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2 -rpath=/software/easybuild/software/GCC/4.9.2/lib64 -rpath=/software/easybuild/software/OpenBLAS/0.2.13-GCC-4.9.2-LAPACK-3.5.0/lib -rpath=/software/easybuild/software/OpenMPI/1.8.4-GCC-4.9.2/lib -rpath=/software/easybuild/software/Ruby/2.1.6/lib -rpath=/software/easybuild/software/ScaLAPACK/2.0.2-gompi-2015a-OpenBLAS-0.2.13-LAPACK-3.5.0/lib -rpath=/software/easybuild/software/hwloc/1.10.0-GCC-4.9.2/lib -rpath=/software/easybuild/software/numactl/2.0.10-GCC-4.9.2/lib sym_str:
/bin/ld.gold: error: cannot open : No such file or directory
collect2: error: ld returned 1 exit status
make: *** [samtools] Error 1

rjeschmi · 2016-01-29T19:29:37Z

[pid 14357] execve("/tmp/eb-A0ozfo/eb-ldwrapper-sTMN3t/ld.gold", ["/tmp/eb-A0ozfo/eb-ldwrapper-sTMN"..., "-plugin", "/software/easybuild/software/GCC"..., "-plugin-opt=/software/easybuild/"..., "-plugin-opt=-fresolution=/tmp/cc"..., "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "-plugin-opt=-pass-through=-lpthr"..., "-plugin-opt=-pass-through=-lc", "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "--eh-frame-hdr", "-m", "elf_x86_64", "-dynamic-linker", "/lib64/ld-linux-x86-64.so.2", ...], [/* 104 vars */] <unfinished ...>
[pid 14357] <... execve resumed> )      = 0
[pid 14360] execve("/usr/bin/tr", ["/usr/bin/tr", "[:upper:]", "[:lower:]"], [/* 103 vars */]) = 0
[pid 14741] execve("/bin/sort", ["sort", "-u"], [/* 103 vars */]) = 0
[pid 14752] execve("/bin/grep", ["grep", "NSC_"], [/* 103 vars */]) = 0
[pid 14751] execve("/bin/env", ["env"], [/* 103 vars */]) = 0
[pid 14753] execve("/bin/ld.gold", ["/bin/ld.gold", "-rpath=/export/easybuild/softwar"..., "-plugin", "/software/easybuild/software/GCC"..., "-plugin-opt=/software/easybuild/"..., "-plugin-opt=-fresolution=/tmp/cc"..., "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "-plugin-opt=-pass-through=-lpthr"..., "-plugin-opt=-pass-through=-lc", "-plugin-opt=-pass-through=-lgcc", "-plugin-opt=-pass-through=-lgcc_"..., "--eh-frame-hdr", "-m", "elf_x86_64", "-dynamic-linker", ...], [/* 103 vars */]) = 0
[pid 14753] open("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/liblto_plugin.so", O_RDONLY|O_CLOEXEC) = 3
[pid 14348] access("/software/easybuild/software/GCC/4.9.2/libexec/gcc/x86_64-unknown-linux-gnu/4.9.2/liblto_plugin.rpo", R_OK) = -1 ENOENT (No such file or directory)

I need to tune the 100 cut forks in the script as well....

rjeschmi · 2016-01-29T19:32:48Z

Do you want me to push my WIP?

rjeschmi · 2016-01-29T19:33:14Z

https://github.com/rjeschmi/easybuild-framework/tree/ld_wrapper

rjeschmi · 2016-02-05T00:37:39Z

So this is sort of working now (though I understand we are going in a different direction).

One odd thing is that using the script I get output like this

 0x000000000000000f (RPATH)              Library rpath: [/export/easybuild/software/ncurses/5.9-foss-2015a/lib -rpath=/export/easybuild/software/zlib/1.2.8-foss-2015a/lib -rpath=/software/easybui
ld/software/FFTW/3.3.4-gompi-2015a/lib -rpath=/software/easybuild/software/GCC/4.9.2/lib -rpath=/software/easybuild/software/GCC/4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2 -rpath=/software/easy
build/software/GCC/4.9.2/lib64 -rpath=/software/easybuild/software/OpenBLAS/0.2.13-GCC-4.9.2-LAPACK-3.5.0/lib -rpath=/software/easybuild/software/OpenMPI/1.8.4-GCC-4.9.2/lib -rpath=/software/easy
build/software/Ruby/2.1.6/lib -rpath=/software/easybuild/software/ScaLAPACK/2.0.2-gompi-2015a-OpenBLAS-0.2.13-LAPACK-3.5.0/lib -rpath=/software/easybuild/software/binutils/2.25-GCC-4.9.3-binutils
-2.25/lib -rpath=/software/easybuild/software/hwloc/1.10.0-GCC-4.9.2/lib -rpath=/software/easybuild/software/numactl/2.0.10-GCC-4.9.2/lib]

So it is detecting the first -rpath= and not seeing the spaces, thus concatenating that all together. That seems wrong...

@ocaisa have you seen anything like that?

ocaisa · 2016-02-05T20:34:38Z

Sorry, I've done a lot of reading but nothing in practice. I can have a little fun with the environment variable approach next week, that'll be easy to take for a test drive.

On 5 Feb 2016 01:37, "Robert Schmidt" <notifications@github.com mailto:notifications@github.com> wrote:

So this is sort of working now (though I understand we are going in a different direction).

One odd thing is that using the script I get output like this

0x000000000000000f (RPATH) Library rpath: [/export/easybuild/software/ncurses/5.9-foss-2015a/lib -rpath=/export/easybuild/software/zlib/1.2.8-foss-2015a/lib -rpath=/software/easybui
ld/software/FFTW/3.3.4-gompi-2015a/lib -rpath=/software/easybuild/software/GCC/4.9.2/lib -rpath=/software/easybuild/software/GCC/4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2 -rpath=/software/easy
build/software/GCC/4.9.2/lib64 -rpath=/software/easybuild/software/OpenBLAS/0.2.13-GCC-4.9.2-LAPACK-3.5.0/lib -rpath=/software/easybuild/software/OpenMPI/1.8.4-GCC-4.9.2/lib -rpath=/software/easy
build/software/Ruby/2.1.6/lib -rpath=/software/easybuild/software/ScaLAPACK/2.0.2-gompi-2015a-OpenBLAS-0.2.13-LAPACK-3.5.0/lib -rpath=/software/easybuild/software/binutils/2.25-GCC-4.9.3-binutils
-2.25/lib -rpath=/software/easybuild/software/hwloc/1.10.0-GCC-4.9.2/lib -rpath=/software/easybuild/software/numactl/2.0.10-GCC-4.9.2/lib]

So it is detecting the first -rpath= and not seeing the spaces, thus concatenating that all together. That seems wrong...

@ocaisahttps://github.com/ocaisa have you seen anything like that?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/651#issuecomment-180125585.

Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,

Prof. Dr. Sebastian M. Schmidt

boegel · 2016-02-13T09:29:23Z

WIP PR by @rjeschmi: #1613

boegel modified the milestones: v1.X, v2.2 Jun 24, 2015

wpoely86 added this to the v2.6.0 milestone Dec 16, 2015

boegel modified the milestones: v2.6.0, v2.7.0 Jan 22, 2016

wpoely86 mentioned this issue Jan 22, 2016

Harden dummy:dummy binutils easybuilders/easybuild-easyconfigs#2228

Merged

boegel modified the milestones: v2.7.0, v2.8.0 Mar 9, 2016

boegel modified the milestones: v2.8.0, v2.x May 10, 2016

boegel modified the milestones: 3.x, 4.x Feb 20, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add support for handling dependencies via RPATH #651

add support for handling dependencies via RPATH #651

boegel commented Jun 2, 2013

fgeorgatos commented Jun 11, 2013

boegel commented Jun 12, 2013

fgeorgatos commented Nov 17, 2013

boegel commented Aug 12, 2015

wpoely86 commented Dec 16, 2015

pforai commented Dec 16, 2015

boegel commented Dec 16, 2015

pforai commented Dec 16, 2015

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Feb 5, 2016

ocaisa commented Feb 5, 2016

boegel commented Feb 13, 2016

add support for handling dependencies via RPATH #651

add support for handling dependencies via RPATH #651

Comments

boegel commented Jun 2, 2013

fgeorgatos commented Jun 11, 2013

boegel commented Jun 12, 2013

fgeorgatos commented Nov 17, 2013

boegel commented Aug 12, 2015

wpoely86 commented Dec 16, 2015

pforai commented Dec 16, 2015

boegel commented Dec 16, 2015

pforai commented Dec 16, 2015

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Jan 29, 2016

rjeschmi commented Feb 5, 2016

ocaisa commented Feb 5, 2016

Prof. Dr. Sebastian M. Schmidt

boegel commented Feb 13, 2016