Modified CUDA easyblock to support wrapper creation #758
Conversation
… use of icpc as a host compiler)
|
Automatic reply from Jenkins: Can I test this? |
| """ | ||
| import os | ||
| import stat | ||
| import shutil |
There was a problem hiding this comment.
@damianam: we try to keep these alphabetical, please reorder?
|
Jenkins: ok to test |
|
@damianam: thanks for the contribution! Please take a look at the remarks, and push an extra commit to your branch to resolve them, let us know if you need any help (you know where to find me ;-)). |
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1411/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
I have applied the suggested changes from this PR and easybuilders/easybuild-easyconfigs#2177. The naming of the host compiler is now provided in the easyconfig, since taking it directly from the framework didn't work (COMPILER_CXX is None, and COMPILER_CUDA_UNIQUE_OPTION_MAP provides an unresolved string 'ccbin="%(CXX_base)s"', since CXX_base is not defined). It is possible now to create a wrapper and give it a host compiler in the easyconfig. Multiple wrappers in a single installation are not supported in the current PR, since in our scheme we will always have CUDA+iccifort, or CUDA+gcc, or CUDA+pgi. In other words, always paired. |
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1428/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1432/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
|
||
| txt = super(EB_CUDA, self).make_module_extra() | ||
|
|
||
| txt += self.module_generator.set_environment('CUDA_HOME', self.installdir) |
There was a problem hiding this comment.
@damianam: this is already there (see method above)?
There was a problem hiding this comment.
Correct, but for some reason it is not working. We should keep the discussion here, instead of the one in easybuilders/easybuild-easyconfigs#2177 since this is an easyblock issue. In any case, why use guesses instead of setting these PATHs manually to well known directories?
There was a problem hiding this comment.
@damianam: the guesses are checked, in the sense that eb will check whether the directory actually exists before including the corresponding statement in the module file.
Additionally, via guesses we'll use prepend-path rather than setenv.
For $CUDA_HOME in particular, I'm not sure if prepend-path makes sense at all. Probably not...
There was a problem hiding this comment.
I don't think it does. CUDA_PATH is not set either in the module. Could it be that prepend-path doesn't work when passing an empty path in the guesses.update? PATH, LD_LIBRARY_PATH and CPATH are defined correctly
There was a problem hiding this comment.
OK, I traced not having $CUDA_HOME or $CUDA_PATH being defined down to a bug in the framework.
It has been there for a while (March 2014), but luckily only affects CUDA.
I'll get that fixed as a last-minute thing in the framework, but this PR needs a bit more love before it can go in, so it'll have to be post v2.5.0.
There was a problem hiding this comment.
Bug that causes the issue with $CUDA_HOME is fixed in easybuilders/easybuild-framework#1519
There was a problem hiding this comment.
@boegel I've made a couple of small changes in the easyblock and easyconfig, so now you can specify a list of host compilers, and the easyblock will create the appropriate wrappers based on that.
…a list of host compilers
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1602/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
@damianam: you'll need to sync this with current |
…er changes in the original EB repo. Update of cuda.py merging changes in the main repo and the ones on the PR for creating a wrapper.
…easyblocks into updated_cuda Conflicts: easybuild/easyblocks/c/cuda.py
|
@boegel I rebased my repo to match the new main develop branch, but it seems I did something wrong :-|. I'll try to fix it before the user meeting. |
|
@boegel I think it is fixed now. I'll do the same with easybuilders/easybuild-easyconfigs#2177 after lunch |
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1638/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
I Could help this a bit during EB meeting, just ask. F. On Tuesday, January 26, 2016, HPC Ugent build bot notifications@github.com
echo "sysadmin know better bash than english"|sed s/min/mins/ |
| def create_wrapper(wrapper_name,wrapper_comp): | ||
| wrapper_f = "%s/bin/%s" % (self.installdir, wrapper_name) | ||
| write_file(wrapper_f, wrapper % wrapper_comp) | ||
| adjust_permissions(wrapper_f, stat.S_IXUSR|stat.S_IRUSR|stat.S_IXGRP|stat.S_IRGRP|stat.S_IXOTH|stat.S_IROTH) |
There was a problem hiding this comment.
wrong indentation (tabs involved)?
|
@boegel I fixed the style, lack of docstrings, etc. |
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1645/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
ping |
|
@damianam: thanks for the ping, need to fix conflict with current |
…locks into updated_cuda Conflicts: easybuild/easyblocks/c/cuda.py
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1671/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1672/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
|
@boegel done |
|
Tested with all existing CUDA easyconfigs and the new one from easybuilders/easybuild-easyconfigs#2177, good to go, thanks @damianam! |
Modified CUDA easyblock to support wrapper creation
This PR modifies the CUDA easyblock to support the optional creation of nvcc wrappers. It adds two extra boolean options for easyconfigs: generate_intel_wrapper and generate_gcc_wrapper. The first one will create a wrapper invcc. The second a wrapper called gnvcc.
This allows to simply tell users "Use invcc to have icpc as a host compiler".
This is a convenience fix that doesn't modify the default behaviour or installation.
It will be followed by another PR in the easyconfigs repo, with an example for iccifort+CUDA.
UPDATE: The naming of the wrappers and how they are created has changed significantly. Take a look at the development of the PR.