add scripts to set up minimal EESSI environment and configure EasyBuild, use init/* scripts in install script, inject RPATH override directory via prepare hook to allow for overriding MPI library #116

ocaisa · 2021-06-22T10:29:07Z

This will configure the relevant rpath override option whenever MPI is part of the toolchain being used to build the software

ocaisa · 2021-06-22T10:35:00Z

Addresses #108

eb_hooks.py

ocaisa · 2021-06-22T10:52:30Z

@boegel I'm not sure how this file should be organised. It looks like it is going to get messy.

ocaisa · 2021-06-22T11:44:19Z

Example when existing value for rpath_override_dirs

[EESSI pilot 2021.03] $ eb -d --disable-cleanup-tmpdir --rpath --rpath-override-dirs=/test/additional:/test/other:: --hooks=./eb_hooks.py OSU-Micro-Benchmarks-5.6.3-gompi-2020a.eb --stop prepare --force 
== Temporary log file in case of crash /tmp/eb-f7rf4lht/easybuild-ly4tamyl.log
== Running parse hook for OSU-Micro-Benchmarks-5.6.3-gompi-2020a.eb...
== Running parse hook for gompi-2020a.eb...
== processing EasyBuild easyconfig /home/alanc/EasyBuild_Git/easybuild-easyconfigs/easybuild/easyconfigs/o/OSU-Micro-Benchmarks/OSU-Micro-Benchmarks-5.6.3-gompi-2020a.eb
== building and installing OSU-Micro-Benchmarks/5.6.3-gompi-2020a...
== fetching files...
== ... (took < 1 sec)
== creating build dir, resetting environment...
== ... (took < 1 sec)
== unpacking...
== ... (took < 1 sec)
== patching...
== ... (took < 1 sec)
== preparing...
== Running pre-prepare hook...
== Updated rpath_override_dirs (to allow overriding MPI family OpenMPI): /test/additional:/test/other:::/cvmfs/pilot.eessi-hpc.org/host_injections/2021.03/software/linux/x86_64/generic/rpath_overrides/OpenMPI
== Running post-prepare hook...
== Resetting rpath_override_dirs to original value: /test/additional:/test/other::
== ... (took 1 secs)
== COMPLETED: Installation STOPPED successfully (took 1 secs)
== Results of the build can be found in the log file(s) /home/alanc/.local/easybuild/build/OSUMicroBenchmarks/5.6.3/gompi-2020a/easybuild/easybuild-OSU-Micro-Benchmarks-5.6.3-20210622.125609.log
== Build succeeded for 1 out of 1
== Keeping temporary log file(s) /tmp/eb-f7rf4lht/easybuild-ly4tamyl.log* and directory /tmp/eb-f7rf4lht.

[EESSI pilot 2021.03] $ grep "Converted RPATH" /tmp/eb-f7rf4lht/easybuild-ly4tamyl.log
== 2021-06-22 12:56:08,923 easyblock.py:2144 DEBUG Converted RPATH override directories ('/test/additional:/test/other:::/cvmfs/pilot.eessi-hpc.org/host_injections/2021.03/software/linux/x86_64/generic/rpath_overrides/OpenMPI') to a list of paths: ['/test/additional', '/test/other', '/cvmfs/pilot.eessi-hpc.org/host_injections/2021.03/software/linux/x86_64/generic/rpath_overrides/OpenMPI']

ocaisa · 2021-06-22T11:45:41Z

Example when no existing value for rpath_override_dirs

[EESSI pilot 2021.03] $ eb -d --disable-cleanup-tmpdir --rpath --hooks=./eb_hooks.py OSU-Micro-Benchmarks-5.6.3-gompi-2020a.eb --stop prepare --force 
== Temporary log file in case of crash /tmp/eb-b6obh53w/easybuild-uqspvn7u.log
== Running parse hook for OSU-Micro-Benchmarks-5.6.3-gompi-2020a.eb...
== Running parse hook for gompi-2020a.eb...
== processing EasyBuild easyconfig /home/alanc/EasyBuild_Git/easybuild-easyconfigs/easybuild/easyconfigs/o/OSU-Micro-Benchmarks/OSU-Micro-Benchmarks-5.6.3-gompi-2020a.eb
== building and installing OSU-Micro-Benchmarks/5.6.3-gompi-2020a...
== fetching files...
== ... (took < 1 sec)
== creating build dir, resetting environment...
== ... (took < 1 sec)
== unpacking...
== ... (took < 1 sec)
== patching...
== ... (took < 1 sec)
== preparing...
== Running pre-prepare hook...
== Updated rpath_override_dirs (to allow overriding MPI family OpenMPI): /cvmfs/pilot.eessi-hpc.org/host_injections/2021.03/software/linux/x86_64/generic/rpath_overrides/OpenMPI
== Running post-prepare hook...
== Resetting rpath_override_dirs to original value: None
== ... (took 1 secs)
== COMPLETED: Installation STOPPED successfully (took 2 secs)
== Results of the build can be found in the log file(s) /home/alanc/.local/easybuild/build/OSUMicroBenchmarks/5.6.3/gompi-2020a/easybuild/easybuild-OSU-Micro-Benchmarks-5.6.3-20210622.125746.log
== Build succeeded for 1 out of 1
== Keeping temporary log file(s) /tmp/eb-b6obh53w/easybuild-uqspvn7u.log* and directory /tmp/eb-b6obh53w.

[EESSI pilot 2021.03] $ grep "Converted RPATH" /tmp/eb-b6obh53w/easybuild-uqspvn7u.log
== 2021-06-22 12:57:44,704 easyblock.py:2144 DEBUG Converted RPATH override directories ('/cvmfs/pilot.eessi-hpc.org/host_injections/2021.03/software/linux/x86_64/generic/rpath_overrides/OpenMPI') to a list of paths: ['/cvmfs/pilot.eessi-hpc.org/host_injections/2021.03/software/linux/x86_64/generic/rpath_overrides/OpenMPI']

ocaisa · 2021-06-22T11:49:56Z

Example when no MPI is used:

[EESSI pilot 2021.03] $ eb -d --disable-cleanup-tmpdir --rpath --hooks=./eb_hooks.py libxc-4.3.4-GCC-9.3.0.eb --stop prepare --force 
== Temporary log file in case of crash /tmp/eb-yjc17doz/easybuild-8he25l7e.log
== Running parse hook for libxc-4.3.4-GCC-9.3.0.eb...
== Running parse hook for GCC-9.3.0.eb...
== Running parse hook for GCC-9.3.0.eb...
== Running parse hook for GCCcore-9.3.0.eb...
== Running parse hook for binutils-2.34-GCCcore-9.3.0.eb...
== Running parse hook for GCCcore-9.3.0.eb...
== Running parse hook for M4-1.4.18.eb...
== Running parse hook for binutils-2.34.eb...
== processing EasyBuild easyconfig /home/alanc/EasyBuild_Git/easybuild-easyconfigs/easybuild/easyconfigs/l/libxc/libxc-4.3.4-GCC-9.3.0.eb
== building and installing libxc/4.3.4-GCC-9.3.0...
== fetching files...
== ... (took < 1 sec)
== creating build dir, resetting environment...
== starting iteration #0 ...
== ... (took < 1 sec)
== unpacking...
== ... (took < 1 sec)
== patching...
== ... (took < 1 sec)
== preparing...
== Running pre-prepare hook...
== Running post-prepare hook...
== ... (took 1 secs)
== COMPLETED: Installation STOPPED successfully (took 3 secs)
== Results of the build can be found in the log file(s) /home/alanc/.local/easybuild/build/libxc/4.3.4/GCC-9.3.0/easybuild/easybuild-libxc-4.3.4-20210622.123239.log
== Build succeeded for 1 out of 1
== Keeping temporary log file(s) /tmp/eb-yjc17doz/easybuild-8he25l7e.log* and directory /tmp/eb-yjc17doz.

eb_hooks.py

…y hooks to inject RPATH override path

let build script also define $EESSI_SOFTWAREPATH, so it can be used by hooks to inject RPATH override path

…openmpi_hook

ocaisa · 2021-06-30T13:40:21Z

EESSI-pilot-install-software.sh

+fi
+
+# make sure we're in Prefix environment by checking $SHELL
+if [[ ${SHELL} = ${EPREFIX}/bin/bash ]]; then


This is what startprefix itself checks, so I think we should do the same

Relying on the environment defined by sourcing init/eessi_environment_variables results in more confusing errors when you're not first in the Gentoo Prefix environment, because then all sorts of things can go wrong, like not having python3 which is used to call eessi_software_subdir.py:

software-layer $ ./EESSI-pilot-install-software.sh >> Setting up environment... >> Determining software subdirectory to use for current build host... ./EESSI-pilot-install-software.sh: line 62: python3: command not found ERROR: no value set for $EESSI_SOFTWARE_SUBDIR ERROR: Failed to determine software subdirectory?!

Another side effect is that the install script now fails hard when the init directory hasn't been populated yet:

>> Setting up environment... >> Determining software subdirectory to use for current build host... /cvmfs/pilot.eessi-hpc.org/2021.06/compat/linux/aarch64/usr/lib/python-exec/python3.9/python3: can't open file '/cvmfs/pilot.eessi-hpc.org/2021.06/init/eessi_software_subdir_for_host.py': [Errno 2] No such file or directory ERROR: no value set for $EESSI_SOFTWARE_SUBDIR ERROR: Failed to determine software subdirectory?!

We've been doing that late, after fully installing the software layer, but I guess doing that early on instead is fine.
We can add a check in the install script to ensure that init has been populated properly, and produce a clear error if not.

That said, I agree that we should clean things up a bit here, and avoid having the same information like $EESSI_PILOT_VERSION in multiple places.

I think I'll pull this apart a bit more, and split off a "minimal_eessi_env script that's sourced by both this install script and eessi_environment_variables, so we can move up this check and get sensible error reporting...

And allow the possibility that the MPI directory may be a symlinked to a version

ocaisa · 2021-07-01T08:57:28Z

eb_hooks.py

+        'rpath_overrides',
+        software_name,
+        # We can't know the version, but this allows the use of a symlink to facilitate version upgrades without removing files
+        'system',


This approach means that you can symlink to a host MPI installation directly (but you'd still need a module to pick up bin).

…than using ':'

…h 'error' function from 'init/eessi_environment_variables'

…n /init being populated

…we're in Gentoo Prefix environment

boegel · 2021-08-27T16:22:04Z

I've pulled things apart a bit more, by also adding a script to configure EasyBuild (configure_easybuild), which is sourced by the install script, and a script to set up a minimal EESSI environment (init/minimal_eessi_env), which is sourced both by the install script and init/init/eessi_environment_variables (which is sourced by init/bash).

So things a more modular now, without having the same information in multiple places...

boegel · 2021-08-27T16:22:46Z

I have also verified that the EasyBuild hook to inject the host_injections RPATH override directories works as intended.

So, time to merge this, thanks for the efforts @ocaisa!

Allow overriding OpenMPI via rpath

626097e

Alan O'Cais added 2 commits June 22, 2021 12:42

Make it general enough for any MPI

b62e0f2

Tidy up

270a533

ocaisa commented Jun 22, 2021

View reviewed changes

eb_hooks.py Show resolved Hide resolved

Tidy up

b783ad0

Was targetting wrong stepgit add .

5176a24

ocaisa changed the title ~~Allow overriding OpenMPI via rpath~~ Allow overriding MPI family via rpath Jun 22, 2021

Alan O'Cais added 4 commits June 22, 2021 12:29

Be more careful when we set/unset an attribute

6aaf63d

Tidy up

15da2a6

Use explicit getattr

9e96fa5

Update eb_hooks.py

0b1861d

ocaisa added the pilot-2021.03 label Jun 23, 2021

boegel added pilot-2021.06 and removed pilot-2021.03 labels Jun 24, 2021

boegel mentioned this pull request Jun 25, 2021

checklist for 2021.06 pilot #118

Closed

14 tasks

boegel requested changes Jun 30, 2021

View reviewed changes

eb_hooks.py Show resolved Hide resolved

boegel and others added 5 commits June 30, 2021 14:18

let build script also define $EESSI_SOFTWAREPATH, so it can be used b…

1bc594b

…y hooks to inject RPATH override path

Merge pull request #2 from boegel/openmpi_hook

fd46408

let build script also define $EESSI_SOFTWAREPATH, so it can be used by hooks to inject RPATH override path

Tidy-up and deduplication

790e1c5

Merge branch 'openmpi_hook' of github.com:ocaisa/software-layer into …

42fb84c

…openmpi_hook

EESSI_SOFTWARE_PATH already set

053073a

ocaisa commented Jun 30, 2021

View reviewed changes

Search in a lib subdirectory

fbe342e

And allow the possibility that the MPI directory may be a symlinked to a version

ocaisa commented Jul 1, 2021

View reviewed changes

Check in both lib and lib64 for overrides

577545e

ocaisa mentioned this pull request Jul 2, 2021

Testing overriding of the MPI in EESSI #121

Closed

boegel added 3 commits August 27, 2021 14:54

Merge branch 'main' into openmpi_hook

b93c859

bump EESSI pilot version in init script, print output message rather …

2d122ec

…than using ':'

cosmetic changes to eb_hooks.py

5f073de

boegel changed the title ~~Allow overriding MPI family via rpath~~ inject RPATH override directory via prepare hook to allow for overriding MPI library Aug 27, 2021

boegel added 4 commits August 27, 2021 17:24

rename 'error' to 'fatal_error' in install script, to avoid clash wit…

fc7d444

…h 'error' function from 'init/eessi_environment_variables'

split of script to define minimal EESSI environment + avoid relying o…

bfc615a

…n /init being populated

source script to set minimal EESSI environment + check early whether …

d0e0706

…we're in Gentoo Prefix environment

add check to see whether EESSI CernVM-FS repository is mounted

44f0fee

boegel changed the title ~~inject RPATH override directory via prepare hook to allow for overriding MPI library~~ add script to set up minimal EESSI environment, use init/* scripts in install script, inject RPATH override directory via prepare hook to allow for overriding MPI library Aug 27, 2021

split off script to configure EasyBuild from install script

4ce9cca

boegel approved these changes Aug 27, 2021

View reviewed changes

boegel merged commit aed951d into EESSI:main Aug 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

add scripts to set up minimal EESSI environment and configure EasyBuild, use init/* scripts in install script, inject RPATH override directory via prepare hook to allow for overriding MPI library #116

add scripts to set up minimal EESSI environment and configure EasyBuild, use init/* scripts in install script, inject RPATH override directory via prepare hook to allow for overriding MPI library #116

Uh oh!

ocaisa commented Jun 22, 2021 •

edited

Loading

Uh oh!

ocaisa commented Jun 22, 2021 •

edited

Loading

Uh oh!

Uh oh!

ocaisa commented Jun 22, 2021

Uh oh!

ocaisa commented Jun 22, 2021 •

edited

Loading

Uh oh!

ocaisa commented Jun 22, 2021 •

edited

Loading

Uh oh!

ocaisa commented Jun 22, 2021 •

edited

Loading

Uh oh!

Uh oh!

ocaisa Jun 30, 2021

Uh oh!

boegel Aug 27, 2021

Uh oh!

ocaisa Jul 1, 2021

Uh oh!

boegel commented Aug 27, 2021

Uh oh!

boegel commented Aug 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

add scripts to set up minimal EESSI environment and configure EasyBuild, use init/* scripts in install script, inject RPATH override directory via prepare hook to allow for overriding MPI library #116

add scripts to set up minimal EESSI environment and configure EasyBuild, use init/* scripts in install script, inject RPATH override directory via prepare hook to allow for overriding MPI library #116

Uh oh!

Conversation

ocaisa commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ocaisa commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ocaisa commented Jun 22, 2021

Uh oh!

ocaisa commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ocaisa commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ocaisa commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

ocaisa Jun 30, 2021

Choose a reason for hiding this comment

Uh oh!

boegel Aug 27, 2021

Choose a reason for hiding this comment

Uh oh!

ocaisa Jul 1, 2021

Choose a reason for hiding this comment

Uh oh!

boegel commented Aug 27, 2021

Uh oh!

boegel commented Aug 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ocaisa commented Jun 22, 2021 •

edited

Loading

ocaisa commented Jun 22, 2021 •

edited

Loading

ocaisa commented Jun 22, 2021 •

edited

Loading

ocaisa commented Jun 22, 2021 •

edited

Loading

ocaisa commented Jun 22, 2021 •

edited

Loading