Add esmf@8.4.1 and mapl@2.35.2 (replace existing versions in unified env) #333
Comments
climbfuji
changed the title
climbfuji
changed the title
climbfuji
changed the title
climbfuji
changed the title
|
@AlexanderRichert-NOAA @mathomp4 @junwang-noaa I started working on this, and quickly realized that we'd be better off waiting for @mathomp4 to finish his PR JCSDA/spack#174. I can add mapl 2.35.2 myself, but it means we'll need to solve conflicts later when merging JCSDA/spack#174. Further, newer mapl versions want We have a pretty tight timeline for the next release. If @mathomp4 thinks that his PR for our spack fork is ready in the next two days (and won't break anything), then lets get it in. Otherwise we could try to copy the mapl and fargparse changes from this PR and deal with the conflicts later, or abandon the idea and stick with mapl 2.22 for this release. (JEDI doesn't need the new mapl functionality, we are fine with 2.22 and ESMF 8.3.0b09 or 8.4.1). |
Another option may be to add the missing variant to enable/disable fargparse from @mathomp4's PR and not worry about fargparse for this release. Then solve conflicts later when JCSDA/spack#174 is ready. |
|
It looks like I am able to build mapl@2.35.2 by turning off the fargparse option. I'll create a draft PR to look at, if that seems to be a safer option than the big PR with all the GEOS packages a day or two before the code freeze, then we may go this route. |
|
Using @mathomp4's draft MAPL package.py and the updated ESMF package.py, I can build the unified stack on hera and successfully build and run the latest UFS with the cpld_control_p8 case (all files match which is impressive!). The only further change was to remove @climbfuji Do you want me to do the PRs for this? If you're already deep into it then go ahead but otherwise I'm happy to make the changes (new mapl version+fargparse dep/variant, and esmf and mapl changes in common/packages.yaml). |
|
Great job, mapl build has to turn off the fargparse.
and netcdf 4.9.1 on acorn should turn off --disable-libxml2
--disable-byterange
@alex If you can do the PRs ,please do it. I can test it with other cases
of UFS. I have tested the UFS aerosols case with mapl. But the case with
the fully coupled one did not work.
…On Tue, Mar 14, 2023 at 2:10 PM Alex Richert ***@***.***> wrote:
Using @mathomp4 <https://github.com/mathomp4>'s draft MAPL package.py and
the updated ESMF package.py, I can build the unified stack on hera and
successfully build and run the latest UFS with the cpld_control_p8 case
(all files match which is impressive!). The only further change was to
remove +pio and add +parallelio for the esmf variants in
common/packages.yaml.
@climbfuji <https://github.com/climbfuji> Do you want me to do the PRs
for this? If you're already deep into it then go ahead but otherwise I'm
happy to make the changes (new mapl version+fargparse dep/variant, and esmf
and mapl changes in common/packages.yaml).
—
Reply to this email directly, view it on GitHub
<#333 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFGWBAQCSKBYLBGZLI3W4CYB3ANCNFSM6AAAAAAQDQDQ7Q>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
|
I think I’ve got it all in my currently open draft PRs, possibly except removing the +pio (please check).
… On Mar 14, 2023, at 12:10 PM, Alex Richert ***@***.***> wrote:
Using @mathomp4 <https://github.com/mathomp4>'s draft MAPL package.py and the updated ESMF package.py, I can build the unified stack on hera and successfully build and run the latest UFS with the cpld_control_p8 case (all files match which is impressive!). The only further change was to remove +pio and add +parallelio for the esmf variants in common/packages.yaml.
@climbfuji <https://github.com/climbfuji> Do you want me to do the PRs for this? If you're already deep into it then go ahead but otherwise I'm happy to make the changes (new mapl version+fargparse dep/variant, and esmf and mapl changes in common/packages.yaml).
—
Reply to this email directly, view it on GitHub <#333 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5C2RJ32S7HYRQUJGKRQJDW4CYB3ANCNFSM6AAAAAAQDQDQ7Q>.
You are receiving this because you were mentioned.
|
|
@climbfuji Okay great, those look good to me as far as this issue is concerned. And yes, @Hang-Lei-NOAA I can add +fismahigh to ufs-weather-model-static/spack.yaml (dap is already disabled, which I believe is what determines libxml2 inclusion). |
|
There was a bug with 4.9.1, beyond the byterange compiler error, wasn’t it? It didn’t work with the UFS when we first tried, that’s why we went back to 4.9.0.
… On Mar 14, 2023, at 1:12 PM, Alex Richert ***@***.***> wrote:
@climbfuji <https://github.com/climbfuji> Okay great, those look good to me as far as this issue is concerned. And yes, +pio will need to get removed in favor of the external pio. Also, can we please change netcdf-c to version 4.9.1 in the same PR? I haven't tested 4.9.0 myself, but apparently there have been problems with it at least w.r.t. to UFS.
@Hang-Lei-NOAA <https://github.com/Hang-Lei-NOAA> I can add +fismahigh to ufs-weather-model-static/spack.yaml (dap is already disabled, which I believe is what determines libxml2 inclusion).
—
Reply to this email directly, view it on GitHub <#333 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5C2RKE5Q35LARIMFLIM2DW4C7ILANCNFSM6AAAAAAQDQDQ7Q>.
You are receiving this because you were mentioned.
|
|
It worked with 4.9.1 for me on hera just now... @Hang-Lei-NOAA @junwang-noaa which netcdf-c version did we end up landing on? |
|
netcdf-c/4.9.1; netcdf-fortran/ 4.6.0 and netcdf-cxx/4.3.1
…On Tue, Mar 14, 2023 at 5:15 PM Alex Richert ***@***.***> wrote:
It worked with 4.9.1 for me on hera just now... @Hang-Lei-NOAA
<https://github.com/Hang-Lei-NOAA> @junwang-noaa
<https://github.com/junwang-noaa> which netcdf-c version did we end up
landing on?
—
Reply to this email directly, view it on GitHub
<#333 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFHKJLQR3OE3POSPZK3W4DNXVANCNFSM6AAAAAAQDQDQ7Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
netcdf-c@4.9.2 was just released with several bug fixes that might be important or at least of interest to us ... https://github.com/Unidata/netcdf-c/releases/tag/v4.9.2 I really don't like updating to a new version so close cutting release branches, but we can always roll back to 4.9.1 if it's needed, and we did have issues with that version before. Any strong objections trying that? |
|
@climbfuji <https://github.com/climbfuji> Could you please add both 4.9.1
and 4.9.2 into spack?
Although I also had some issues with 4.9.1, the UFS team will use it for
this upgrade step.
…On Tue, Mar 14, 2023 at 10:20 PM Dom Heinzeller ***@***.***> wrote:
***@***.*** was just released with several bug fixes that might be
important or at least of interest to us ...
https://github.com/Unidata/netcdf-c/releases/tag/v4.9.2 I really don't
like updating to a new version so close cutting release branches, but we
can always roll back to 4.9.1 if it's needed, and we did have issues with
that version before.
Any strong objections trying that?
—
Reply to this email directly, view it on GitHub
<#333 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AKWSMFCE5H6LBTQZIIAWBTTW4ERQDANCNFSM6AAAAAAQDQDQ7Q>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
We need to choose one to build everything with. I can add 4.9.2 to the spack package, so that spack knows about it, but we can only install one. I’ll give it a shot with 4.9.2.
… On Mar 14, 2023, at 8:52 PM, Hang-Lei-NOAA ***@***.***> wrote:
@climbfuji <https://github.com/climbfuji> Could you please add both 4.9.1
and 4.9.2 into spack?
Although I also had some issues with 4.9.1, the UFS team will use it for
this upgrade step.
On Tue, Mar 14, 2023 at 10:20 PM Dom Heinzeller ***@***.***>
wrote:
> ***@***.*** was just released with several bug fixes that might be
> important or at least of interest to us ...
> https://github.com/Unidata/netcdf-c/releases/tag/v4.9.2 I really don't
> like updating to a new version so close cutting release branches, but we
> can always roll back to 4.9.1 if it's needed, and we did have issues with
> that version before.
>
> Any strong objections trying that?
>
> —
> Reply to this email directly, view it on GitHub
> <#333 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AKWSMFCE5H6LBTQZIIAWBTTW4ERQDANCNFSM6AAAAAAQDQDQ7Q>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
—
Reply to this email directly, view it on GitHub <#333 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5C2RI4BYIENMKNTTHVP43W4EVGBANCNFSM6AAAAAAQDQDQ7Q>.
You are receiving this because you were mentioned.
|
|
Compiled and ran UFS WM successfully on Hera with 4.9.2 using unified env, so between that and 4.9.0 I'd opt for 4.9.2. |
|
Thanks @AlexanderRichert-NOAA ! I am trying that version this morning, too. |
|
Note on the GEOS side (which of course isn't quite ready for spack-stack): Since we are a netCDF-Fortran shop, we'll probably stick with netCDF-C 4.9.0 for the near future. From the announcement:
I always wait until I hear from the team that netCDF-Fortran is "good". I mean, I'm sure it does work, but I'm pretty conservative with netCDF. (Meanwhile, ESMF betas galore! 😄 ) |
Please describe the package or library you would like to add to spack-stack.
See NOAA-EMC/hpc-stack#485 for more details.
What applications will be using this package or library?
UFS
Is there already a package or library in spack-stack that provides this, or related, functionality?
Earlier versions of mapl
Additional context
n/a
Will This Package be Needed in a NOAA Operational Application?
Yes
WCOSS System Software Request Checklist
See NOAA-EMC/hpc-stack#485 (if there is no checklist in hpc-stack, then let's ask the requester to add one).
The text was updated successfully, but these errors were encountered: