-
Notifications
You must be signed in to change notification settings - Fork 284
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
generic easyblock for Cray toolchains #766
Conversation
Example easyconfig:
resulting module file:
|
cc @pforai |
Easyblocks unit test suite FAILed. See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1461/console for more details. Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do. |
@wpoely86: please review? |
prgenv_unloads.append(self.module_generator.unload_module(prgenv).strip() + '\n') | ||
|
||
comment = self.module_generator.comment("next, load toolchain components") | ||
txt = '\n'.join(prgenv_unloads) + '\n' + comment + txt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add \n
between comment
and txt
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that's already there implicitly (since I'm not stripping the return value of .comment()
), it would result in a double newline in the generated module file
Easyblocks unit test suite FAILed. See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1471/console for more details. Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do. |
Easyblocks unit test suite FAILed. See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1476/console for more details. Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do. |
cleaner output after feedback by @rtmclay and adjusting easybuilders/easybuild-framework#1506 accordingly:
|
Jenkins: test this please |
Easyblocks unit test suite FAILed. See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1490/console for more details. Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do. |
Jenkins: test this please |
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1496/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
Looks better now, why no swap instead of the unload/load sequence? |
@pforai: a |
@gppezzi: I'd love to see an easyconfig PR that using |
When
But trying to build HPL on top of this fails. With not having a PrgEnv loaded, we get:
When we do have a PrgEnv loaded, we get:
cc @gppezzi |
Actual problem in last case:
|
The problem seems to be that the modules tool does not respect the The @rtmclay: is there a way to fix this? |
I don't think it is safe to load the PrgEnv-gnu/5.2.40 and then unload gcc and reload it. But to really tell what is going on, you'll need to find the exact place where in the CrayGnu modulefile it is failing. I think it will be inside one of the Cray modules and not in the one you are showing above. |
@rtmclay: well, it's not failing in the sense that loading this module fails, but we're getting an environment that isn't what we think it should be. How would you pinpoint the exact problem here? Also, why do things work when I have the I'm seeing the same problem when using |
@rtmclay gave a good hint: unloading PrgEnv-gnu and then reloading it seems to be the cause of this problem. Strangely enough it works for the first time you load CrayGNU but it ruins your environment if you load it twice (and that' s probably what EB is internally doing in this case?). I tried this workaround in order to avoid unloading it and by far this seems to work:
|
fix for PrgEnv-* unloading problem
Easyblocks unit test suite FAILed. See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1535/console for more details. Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do. |
@gppezzi: we should pick this up again, to get it merged for EB v2.6.0... |
Jenkins: test this please |
Easyblocks unit test suite FAILed. See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1606/console for more details. Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do. |
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1608/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1609/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
@gppezzi: I think this is good to go now, can you please retest/review/confirm? |
lgtm HPL basic tests passed on both Piz Daint and Piz Dora (see #2222). I did some basic testing and I the environment problems caused by reloading PrgEnv-* seem to be fixed with this version. I've also successfully used this easyblock to compile Python, including numpy on top of libsci. Thanks a lot @boegel for all the support!! |
@gppezzi: thanks for the feedback, I see no reason to hold this back any further It would be great if @pforai is able to test this on another (non-CSCS) Cray system indeed, with the easyconfigs available in easybuilders/easybuild-easyconfigs#2222, but he can also do that with EasyBuild v2.6.0 very soon. ;-) |
Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1618/console for more details). This pull request is now ready for review/testing. Please try and find someone who can tackle this; contact @boegel if you're not sure what to do. |
generic easyblock for Cray toolchains
After hitting a couple of nasty bugs and after discussion with Cray, @pforai and @boegel, here's the new proposal for the module file.
|
(requires
easybuilders/easybuild-framework#1506)required for stable Cray support (cfr. easybuilders/easybuild-framework#1390)
cc @gppezzi