Roll out spack-stack-1.6.0 #924
Comments
climbfuji
added
INFRA
|
@climbfuji can you write a list of updated libraries in release/1.6.0? I think the library team (@Hang-Lei-NOAA @AlexanderRichert-NOAA ) need to test them on acorn and once confirmed, make requests for installation on wcoss2, which could take some time to finish. We are trying to avoid the libraries to diverge on wcoss2. |
|
Can EPIC or EMC do this please? @AlexanderRichert-NOAA @ulmononian |
|
Otherwise I'll do that when I draft the release notes in a few days |
|
Yeah I can do that. @junwang-noaa I assume we're just talking about UFS WM-related libraries? |
|
Thanks very much @AlexanderRichert-NOAA ! |
|
@AlexanderRichert-NOAA Yes. Also would you please install the spack-stack 1.6.0 on acorn for use to do testing? Once it is done, we will ask @Hang-Lei-NOAA to make requests to install the new libraries on wcoss2. Thanks |
|
@climbfuji you can add me in description for gaea-c5, hera and jet. |
Thanks @RatkoVasic-NOAA ! |
|
@climbfuji - please add me in description for NOAA Parallelworks (AWS, GCloud, Azure) |
Thanks very much @natalie-perlin! I had no problem rolling it out on gcloud, no site config updates needed. So hopefully this works just fine for you out of the box as it did for me. |
|
@junwang-noaa:
Everything else will be the same as in the current ufs_common.lua: |
|
Acorn is done (no duplicates, no missing shared libs, permissions good): /lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.6.0/envs/unified-env |
|
@RatkoVasic-NOAA Can you please provide the locations of the |
|
@climbfuji @AlexanderRichert-NOAA may I ask what new feature is included in the netcdf-fortran 4.6.1? There are several libraries are depends on netcdf, those libraries need to be rebuilt on wcoss2 with hpc-stack. It may take longer time to get all these libraries installed. |
@climbfuji
II) gsi-addon-env: not sure. What is procedure for that? III) Changes: |
|
@junwang-noaa looking at the release notes, it appears the main functional difference between 4.6.0 and 4.6.1 is a fix in the quantization. |
|
@RatkoVasic-NOAA re: II, see the instructions at https://spack-stack.readthedocs.io/en/latest/AddingTestPackages.html#add-test-packages. Basically, to install gsi-addon-env, use the |
|
@AlexanderRichert-NOAA are you doing |
|
@climbfuji - All of the stacks are installed in |
|
Thanks @natalie-perlin ! |
|
I see that 1.6.0 was installed on f2 on Gaea-C5. Was this intentional? I thought it was supposed to be installed on f5. |
|
1. f5 is not ready for use. I have been given early access but it’s not ready for prime time yet.
2. Software are not to be installed on f5. We have requested a project space in the right place. We will reinstall spack-stack on that space when it is ready for us and then switch over
… On Jan 11, 2024, at 5:53 AM, David Huber ***@***.***> wrote:
I see that 1.6.0 was installed on f2 on Gaea-C5. Was this intentional? I thought it was supposed to be installed on f5.
—
Reply to this email directly, view it on GitHub <#924 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5C2RN373RVWEQXSXPZ4U3YN7OF3AVCNFSM6AAAAABBF36NPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGEYDMOBUGA>.
You are receiving this because you were mentioned.
|
|
Thanks for clarifying, @climbfuji! |
|
@AlexanderRichert-NOAA I tested the spack-stack 1.6.0 with the updated libraries. The RT passed on acorn. As for netcdf-fortran 4.6.1, @edwardhartnett may I ask what the major updaters are in netcdf-fortran 4.6.1? I remember NCO may have some reservation on installing minor updates for major libraries that have many dependencies. Not sure if they will approve the request if we ask for 4.6.1. Thanks |
|
The only thing in netcdf-fortran-4.6.1 is some of the quantize constants
are defined, which were undefined in 4.6.0. However, I believe Dusan
already wrote the code, and he just defined them in his code, which works
fine.
So we can live without the netcdf-fortran-4.6.1 release if that's helpful.
Ed
…On Thu, Jan 11, 2024 at 8:21 AM Jun Wang ***@***.***> wrote:
@AlexanderRichert-NOAA <https://github.com/AlexanderRichert-NOAA> I
tested the spack-stack 1.6.0 with the updated libraries. The RT passed on
acorn.
@Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> would you please make
requests to install the new libraries: ESMF/8.6.0, FMS/2023.04, crtm/
2.4.0.1 and sp/2.5.0.
As for netcdf-fortran 4.6.1, @edwardhartnett
<https://github.com/edwardhartnett> may I ask what the major updaters are
in netcdf-fortran 4.6.1? I remember NCO may have some reservation on
installing minor updates for major libraries that have many dependencies.
Not sure if they will approve the request if we ask for 4.6.1. Thanks
—
Reply to this email directly, view it on GitHub
<#924 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AJIOMMH2OM7ZULZNOEXFSRTYN77QPAVCNFSM6AAAAABBF36NPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGQYDGNRTGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
--
Edward Hartnett
CIRES/NOAA EMC
|
|
So, typically patch version updates are bug fixes of existing versions that address real problems such as undefined constants for the above example, or security vulnerabilities, etc, without adding new functionality that have greater potential to break things. I think it would be highly advisable to revert the policy and always go for patch level updates. For spack-stack in general, I suggest we always opt to use the bug-fixed version of a package with otherwise the same functionality. |
|
From @malloryprow:
Could these be populated into gsi-addon? They are needed for the EMC_verif-global repo. |
THat will need to be done via the gsi-addon-env template |
|
@climbfuji since the gsi-addon-env uses and subsequently shouldn't all of the |
|
No - only the ones that are requested by gsi-addon-env for which the gsi-addon-env requirements match existing packages in unified-env |
|
Alright, thanks for that info. I will update and test the template and handle any needed installations in another issue. |
|
@jun Wang - NOAA Federal ***@***.***> Please test these libs on
acorn before we start a code deliver:
/lfs/h1/emc/nceplibs/noscrub/hpc-stack/libs/hpc-stack/modulefiles/mpi/intel/19.1.3.304/cray-mpich/8.1.9/
esmf/8.6.0
mapl/2.40.3-esmf-8.6.0
fms/2023.04
Other libs were also installed previously.
…On Thu, Jan 11, 2024 at 11:38 AM David Huber ***@***.***> wrote:
Alright, thanks for that info. I will update and test the template and
handle any needed installations in another issue.
—
Reply to this email directly, view it on GitHub
<#924 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFEQ5SNY2QIIRQ5K6M3YOAIPZAVCNFSM6AAAAABBF36NPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGU2DKMRYGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@Hang-Lei-NOAA Can I ask an unrelated question on how boost is installed on WCOSS2? |
|
@dominikus Heinzeller - NOAA Affiliate ***@***.***> The boost
is not delivered by us. The current boost is there when GDIT sets up the
machine.
…On Thu, Jan 11, 2024 at 11:57 AM Dom Heinzeller ***@***.***> wrote:
@Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> Can I ask an unrelated
question on how boost is installed on WCOSS2?
—
Reply to this email directly, view it on GitHub
<#924 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFCEKICBB5TIXALI3FLYOAKZBAVCNFSM6AAAAABBF36NPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGU3TQNRUGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@Hang-Lei-NOAA would you please provide a module file with the HPC-stack libraries on acorn? Currently the model only has spack stack module file on acorn: https://github.com/ufs-community/ufs-weather-model/blob/develop/modulefiles/ufs_acorn.intel.lua I am not sure how to combine these libraries with the libraries you installed with HPC stack. |
|
Sure, no problem.
…On Thu, Jan 11, 2024 at 12:23 PM Jun Wang ***@***.***> wrote:
@Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> would you please
provide a module file with the HPC-stack libraries on acorn? Currently the
model only has spack stack module file on acorn:
https://github.com/ufs-community/ufs-weather-model/blob/develop/modulefiles/ufs_acorn.intel.lua
I am not sure how to combine these libraries with the libraries you
installed with HPC stack.
Thanks
—
Reply to this email directly, view it on GitHub
<#924 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFBTEPWR37UCNDB3HEDYOANYTAVCNFSM6AAAAABBF36NPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBXGYZDANJWGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
All done! |
|
@AlexanderRichert-NOAA Apologies for coming in late on this conversation, but is it possible to add a few additional install requests for WCOSS2 needed by the global workflow? In particular, we would need gsi-ncdiag@1.1.2 built against netcdf-fortran@4.6.1 I'm happy to open another issue if that's what's needed here. |
|
@Hang-Lei-NOAA can you request those on WCOSS2? |
|
@AlexanderRichert-NOAA I will handle these. @DavidHuber-NOAA nco@5.0.6 I deliver it before new year. Has been installed by GDIT on wcoss2. |
|
@Hang-Lei-NOAA Thank you very much for the explanations and for working on this! |
Is your feature request related to a problem? Please describe.
This issue captures the necessary tasks for rolling out spack-stack-1.6.0.
Installation procedure:
unified-envfromrelease/1.6.0branch of spack-stack (and spack submodule)gsi-addon-envon selected platforms as chained environment (see list of platforms below)gsi-addon-envDescribe the solution you'd like
gsi-addon-env(@climbfuji)fms@2023.02.01,sp@2.3.3, andcrtm@2.4.0to unified-env because of Spack-stack 1.5.1, ESMF 8.5.0, FMS 2023.02.01 + Remove Gaea C4 + Fix build system to allow CMAKE_<COMPILER>_FLAGS to be specified for submodules #2052 ufs-community/ufs-weather-model#2013 (@climbfuji)gsi-addon-env(@climbfuji)gsi-addon-env(@climbfuji)gsi-addon-env: (@RatkoVasic-NOAA)gsi-addon-env: (@RatkoVasic-NOAA)skylab-7.0.0-redhat8in JCSDA USAF account, us-east-2 is ami-091ad0584d0400762skylab-7.0.0-redhat8in JCSDA USAF account, us-east-1 is ami-01147e0e00b99cbdfspack-stack-1.6.0release/1.6.0to tagspack-stack-1.6.0of spack submodule in container recipes and.gitmodules1.6.0andspack-stack-1.6.0release/1.6.0to1.6.0spack-stack-v1(containingskylab-3.0.0releases and older) from all supported platforms. Check if any of the legacy thirdparty packages can be removed. Configure remotes and check out final tags1.6.0/spack-stack-1.6.0inspack-stack-1.6.0directory.AcornJCSDA CI containersn/ajcsda_emc_spack_stack/develop:Additional context
n/a
The text was updated successfully, but these errors were encountered: