Add CI on LC Gitlab #582
Add CI on LC Gitlab #582
Conversation
|
LGTM |
| @@ -120,7 +120,7 @@ config: | |||
| # - If set to 16 on a 4 cores machine `spack install` will run `make -j4` | |||
| # - If set to 16 on a 18 cores machine `spack install` will run `make -j16` | |||
| # If not set, Spack will use all available cores up to 16. | |||
| build_jobs: 8 | |||
| build_jobs: 64 | |||
There was a problem hiding this comment.
this sets the max for all builds, if you can also change per platform by adding a config.yaml
There was a problem hiding this comment.
I changed it to take advantage of the full node allocated to build, on both machines.
Yes, it’s the "max". So, if less cores are available (quartz 36), build will use all cores (-j 36), not 64.
Isn’t that OK of every machine?
There was a problem hiding this comment.
All cores would be bad if if someone builds on a login node.
There was a problem hiding this comment.
Limiting the number of cores is a shame in CI context. Since someone could potentially build on a login node on any machine, should I go back to 8 cores? I’m not even sure a process using all cores would be allowed on a login node.
Could we "encourage" people to build on allocated resource? I can document it.
There was a problem hiding this comment.
In other CI contexts (azure, travis) we have very limited resources, in other projects we have run out of ram even when using multi threaded builds.
You can always use a custom config for these CI tests?
There was a problem hiding this comment.
I see your point, and here is where I am in my reflexions:
- Definitely, 64 should no be applied to all contexts, as is the case right now
-> Use specific config for quartz and lassen - But, even on quartz and lassen, the 64 is not desirable when building on login node
-> Either no build on login node, or patch config.yaml on the fly for CI.
I we choose to patch on the fly, we don’t even need a specific config for lassen and quartz.
Lastly, I don’t see another way to customize spack config for CI context, because spack config is determined by uberenv using the machine info "only".
There was a problem hiding this comment.
but the ci script is always running CI, so it could select a config-ci.yaml, or something?
Not picky about the CI solution, but i do want to avoid 64 (max) for all configs.
Thanks! I’m glad you like it! |
|
LGTM |
|
Failures are due to fetching instabilities in Spack. Failing tests passed with previous commit, and change introduced have no impact outside Gitlab CI, which passed. |
|
thanks! |
This PR is the "Add CI" part of #517.
Uberenv has been only slightly modified to fix a bug with mirroring.
The spack configuration has been updated based on Ascent one.
The failing jobs in CI are marked as "Allowed failure" when the cause is identified as beyond the scope of this PR.