Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend framework to enable modules to ship RPATH wrappers #3918

Open
9 tasks
stderr-enst opened this issue Dec 17, 2021 · 2 comments
Open
9 tasks

Extend framework to enable modules to ship RPATH wrappers #3918

stderr-enst opened this issue Dec 17, 2021 · 2 comments
Labels
EESSI Related to EESSI project enhancement
Milestone

Comments

@stderr-enst
Copy link

stderr-enst commented Dec 17, 2021

In the EESSI hackathon we started to work on an issue when building Software with RPATH-enabled compiler modules.
As a user, I don't really want to untangle the RPATHs for the modules library, when I am building my own code. So we thought the already existing RPATH wrappers could be automatically loaded when loading an RPATH enabled compiler module. (configurable as an easyconfig-option!)

In this issue I want to track a couple of changes that are required on the framework side, to allow modules to ship RPATH wrapper scripts when they are loaded.

There are a couple of open points to address:

  1. modify module_generator.py (@boegel):
    • Add conditional statement for environment variables. Then we can ignore rpath wrappers while loading a module if something like DISABLE_RPATH_WRAPPERS is set
    • Ensure that wrappers are not put into $PATH if module is used within EasyBuild
  2. modify prepare_rpath_wrappers() in toolchain.py (@stderr-enst):
    • Add function parameter to prepare_rpath_wrappers to install wrappers in a caller-given directory
    • Return directory path where wrappers were installed
    • Add option to put all wrapper scripts in a single directory (with no subdirs like gcc_wrapper/gcc)
    • Don't write rpath_wrapper_log to tempfile.gettempdir() if debug is set. This location does not necessarily exist if a module is loaded with the wrapper scripts in place
  3. modify rpath_wrapper_template.sh.in:
    • Add option to read env variables RPATH_INCLUDE_DIRS and RPATH_FILTER_DIRS variables, to include/exclude directories in wrapper script (as comma-separated list) at runtime
    • Figure out a useful way to treat logging in rpath_wrapper_template.sh.in. Otherwise this will try to log to possibly non-existing /tmp/-dirs or similar
  4. Add create_rpath_wrappers(), probably in toolchain.py(?):
    • Refactor create_rpath_wrappers from easybuild-easyblocks/pull/2638 into a more generic location within framework. This is probably used for other compilers in the future as well

Related PRs:

@ocaisa
Copy link
Member

ocaisa commented Dec 17, 2021

Need to ensure that the CUDA stubs directory is excluded from the included rpaths

@stderr-enst
Copy link
Author

Need to ensure that the CUDA stubs directory is excluded from the included rpaths

This should already happen in prepare_rpath_wrappers(): https://github.com/easybuilders/easybuild-framework/blob/develop/easybuild/tools/toolchain/toolchain.py#L964
If the rpath wrapper script also listens to environmental variables like RPATH_FILTER_DIRS, you could also exclude them even if the module is already build and loaded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
EESSI Related to EESSI project enhancement
Projects
None yet
Development

No branches or pull requests

3 participants