Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ccpp-framework errors with Python 3.11 #456

Merged
merged 2 commits into from
Jan 6, 2023

Conversation

climbfuji
Copy link
Collaborator

@climbfuji climbfuji commented Nov 23, 2022

Fix Python 3.11 errors as described in #457

With these changes, the error from

cd tests
PYTHONPATH=$PWD/../scripts/parse_tools:$PWD/../scripts:$PYTHONPATH python3.11 test_metadata_parser.py

is fixed. I asked @DusanJovic-NOAA to test this branch with the ufs-weather-model on his Fedora 37 machine, which uses Python 3.11 as default, and I would like to ask @mkavulich @grantfirl @dustinswales to test this branch with their versions of Python in the CCPP-SCM, please.

As noted in #457, I cannot run the unit tests in ccpp-framework that require pytest. I ran the following unit tests successfully with Python 3.11:

cd tests
PYTHONPATH=$PWD/../scripts/parse_tools:$PWD/../scripts:$PYTHONPATH python3.11 test_metadata_parser.py

I then ran all unit tests successfully with the native Python 3.10 installation on Ubuntu 22.04:

cd tests
PYTHONPATH=$PWD/../scripts/parse_tools:$PWD/../scripts:$PYTHONPATH python3.11 test_metadata_parser.py
PYTHONPATH=$PWD/../scripts/parse_tools:$PWD/../scripts:$PYTHONPATH python3.11 test_mkstatic.py
cd ..
cd test
./pylint_test.sh # average code rating about 8.5/10
./run_doctest.sh
./run_tests.sh

User interface changes?: No

Fixes: #457 (to be confirmed)

Testing: see above

…work with Python 3.11: global inline flags in the middle of regular expressions are deprecated since Python 3.6, and no longer supported in Python 3.11
@climbfuji
Copy link
Collaborator Author

Update. @DusanJovic-NOAA reported that this branch allows him to compile the ufs-weather-model with Python 3.11.

@dustinswales
Copy link
Collaborator

dustinswales commented Nov 23, 2022

@climbfuji I was able to replicate the error with python3.11, and confirm that this fixes the error:

(scm_py311) framework/tests> setenv PYTHONPATH $PWD/../scripts
(scm_py311) framework/tests> setenv PYTHONPATH $PYTHONPATH:$PWD/../scripts/parse_tools
(scm_py311) framework/tests> python test_metadata_parser.py
...
File "/glade/work/dswales/conda-envs/scm_py311/lib/python3.11/re/_parser.py", line 841, in _parse
raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 3

(scm_py311) framework/tests> cd ../../
(scm_py311) ccpp/framework> git remote add dom https://github.com/climbfuji/ccpp-framework
(scm_py311) ccpp/framework> git fetch dom
(scm_py311) ccpp/framework> git checkout dom/bugfix/python311_global_flags
(scm_py311) ccpp/framework> cd framework/tests
(scm_py311) framework/tests> python test_metadata_parser.py
(scm_py311) framework/tests>

@climbfuji
Copy link
Collaborator Author

Thanks, @dustinswales!

Dusan also ran the unit tests for the framework on his Fedora system, and they pass just like on my box with Python 3.10. I am currently running the ufs-weather-model regression tests on hera with intel against this branch.

@climbfuji climbfuji changed the title DRAFT: Fix ccpp-framework errors with Python 3.11 Fix ccpp-framework errors with Python 3.11 Nov 23, 2022
@DusanJovic-NOAA
Copy link
Collaborator

I successfully compiled ufs-weather-model on Fedora 37 (which uses python 3.11)

@climbfuji
Copy link
Collaborator Author

Note. Full regression tests for ufs-weather-model passed on Hera with Intel.

Copy link
Collaborator

@gold2718 gold2718 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking care of this!

@climbfuji
Copy link
Collaborator Author

Thanks for taking care of this!

Good to hear from you :-)

@jkbk2004
Copy link

jkbk2004 commented Jan 6, 2023

all tests are done on ufs-community/ufs-weather-model#1522, can we start process to merge in this pr?

@climbfuji climbfuji merged commit 1b6352f into NCAR:main Jan 6, 2023
@climbfuji climbfuji deleted the bugfix/python311_global_flags branch January 6, 2023 17:39
@climbfuji
Copy link
Collaborator Author

@jkbk2004 Merged, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ccpp-framework not working with Python 3.11
6 participants