-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spack-stack METplus fails with PosixPath errors #2091
Comments
Can we try using metplus 5.1.0 (and whichever met version should go with it) within GW and see if that fixes things? |
Sure, I can give that a whirl. |
I ran a test Grid2Grid job on Orion with met/11.1.0 and metplus/5.1.0. I first had to make a couple adjustments run_verif_global_in_global_workflow.sh, which I have captured in this branch. This resulted in a number of errors when parsing MET configuration files. Possibly related, copious warnings about unused configuration settings were reported as well, though I seem to recall some of these were present in past, successful runs. I am guessing based on the log output that the format of MET configuration files has changed since version 9.1.3. The log file can be found here: /work/noaa/global/para/com/ss_151/logs/2021111000/gfsmetpg2g1.log. To isolate a single error, the following grid_stat command
failed to parse the file
@malloryprow Would you mind taking a look at the log files and see if I am interpreting this correctly? |
Yeah, I was playing around with this earlier. The change from METplus v3 to METplus v4 is a big one. What worked in METplus v3 won't work in METplus v4. All the METplus config files will need to be updated accordingly. METplus v4 is much similar to METplus v5. |
While it is preferred to get this working with MET v9.1.3 and METplus v3.1.1, if we need to move forward with using MET v11.1.0 and METplus v5.1.0 I can get everything updated. It would probably take me a week or two to get everything changed and tested. Please let me know ASAP what route we want to take. |
@malloryprow It would be nice to have all of the libraries/repositories moved over to spack-stack and it looks to me like that would mean we would need to upgrade to METplus 5.1. I would like to try playing around with METplus 3.1.1 and Python 3.10.8 a little more, which I will try tomorrow, just to check if we the If we do need to do an upgrade, I am happy to run tests for you from the global workflow. |
Thanks for giving the older versions another go. Keep me updated! |
I've been thinking about this more and the upgrades would be okay for stats, but the plotting capabilities that EMC_verif-global uses from METplus v3.1.1 are no longer supported in more recently version of METplus. |
I tracked this issue down to some specific versions of Python starting in mid-2021 here: python/cpython#88227. There is a fix for it in newer versions of Python, so we can probably ask the spack-stack team to get a newer version of Python in spack-stack/1.6.0. It looks like further down in the comments of that issue that version 3.10.11 (or maybe 3.11?) still had this issue, so we may need something newer than that, maybe 3.10.12. FYI @AlexanderRichert-NOAA @climbfuji. |
I'll open a spack-stack issue. |
Thank you @DavidHuber-NOAA! |
@aerorahul is going to set up a meeting next week with the relevant players to discuss a way forward should we not be able to make this work for METplus 3.1.1 (and maybe even if we are, for medium-term plans). |
This will be resolved with the upgrade to spack-stack 1.6.0 (#2195). |
This can be closed with #2239 merged. |
closed via #2195 |
What is wrong?
When running metplus/3.1.1 built with spack-stack within the global-workflow, multiple Python crashes are reported by PosixPath. For instance:
What should have happened?
The METplus jobs should be able to run without error.
What machines are impacted?
Hera
Steps to reproduce
git clone git@github.com:DavidHuber-NOAA/global-workflow -b spack-stack
DO_METP=YES
gfsmetpg2g
, etc kicks off and check the logs.Additional information
The Python version used by spack-stack/1.5.1 is 3.10.8 which is much newer than is used on WCOSS2 (3.8.6). This was only tested on Hera, but suspected to be an issue on all spack-stack systems.
Do you have a proposed solution?
No response
The text was updated successfully, but these errors were encountered: