Discontinue Python 2.7 support in the next version of CMOR

[mauzey1]

With the approaching end-of-life for Python 2, should we consider dropping CMOR's support for Python 2.7 for the release of CMOR 3.6.0?

[durack1]

@mauzey1 would it be reasonable to continue support until a 2020 release, maybe 3.7.0? This requires no additional effort on your part right?

We could add a note to the release info for 3.6.0 (see cmor/releases/tag/3.5.0) that python2 support will be dropped in a 2020 version

[mauzey1]

@durack1 Okay, we will hold off on discontinuing Python 2.7 until a future release in 2020.

Should we consider adding Python 3.8 support to 3.6.0?

[durack1]

@mauzey1 totally, I think we should make sure that CMOR3.6.0 supports all current Python releases, so Py2.7.17, Py3.5.x, 3.6.x, 3.7.x and 3.8.0

It seems there are many current variants available from https://www.python.org/downloads/

Is this much additional work?

[matthew-mizielinski]

An interesting question -- I'd suggest you ask data processors using CMOR which version they use and whether they use python3. As PrePARE is part of the publication process (or should be) it might be worth checking with the ESGF node owners as well.

From the Met Office perspective I don't expect to have made the transition from python2.7 to python3.x until mid 2020 at the earliest. We currently use CMOR v3.4 and are looking at v3.5 now.

[taylor13]

This is important input. Looks like the community may not be a nimble as we think.

Why don't @mauzey1 @durack1 and @taylor13 meet and discuss before making a decision on Python 2.7

[durack1]

@matthew-mizielinski thanks for chiming in, it's useful to have "someone on the ground" who can provide direct feedback on usage.

Now I am curious, considering Python2.7 is due for "death" in just over a month (at least lack of support) what is the MOHC plan to migrate to Python3.x?

For your institutional point of view, what would you want us to provide regarding Python2.7 support, considering Python2.7 itself will no longer be supported after 1st January 2020?

For context, here is a blurb on Python2 support in linux variants going forward:

...
For Ubuntu 18.04 LTS, Python 2 is not part of the installation image. Invoking python will
either remind users how to install Python 2 or will note the existence of Python 3 if it is
installed.
...
Debian has made the decision to ship Python 2 with Debian 10 ("buster"), which will be
released in mid-2019; that means Python 2 will be supported in Debian well past its
2020 end of life. For Ubuntu, the plan for the 20.04 LTS release (scheduled 23 April
2020) has Python 2 removed from the main repository. That means it will not be
supported by Canonical, so the community will need to pick it up if it is to continue
at that point.

For RHEL specifically (the standard LLNL image), it seems RHEL8 supports both Python2/3 at release time https://developers.redhat.com/blog/2019/05/07/what-no-python-in-red-hat-enterprise-linux-8/ (with Python3.6 the factory default), not sure what is going to happen in the patch releases post 1st January 2020.

[mauzey1]

@durack1 @taylor13 I agree that we should have a discussion at some point about Python support in future CMOR releases.

Maybe we should have a survey of CMOR users on how they use the software, which versions they are on, etc.

[durack1]

@mauzey1 a google form questionnaire is something we could send around if it was developed, not sure I have the right information now to know what questions would be useful to ask

[matthew-mizielinski]

@matthew-mizielinski thanks for chiming in, it's useful to have "someone on the ground" who can provide direct feedback on usage.

Now I am curious, considering Python2.7 is due for "death" in just over a month (at least lack of support) what is the MOHC plan to migrate to Python3.x?

For your institutional point of view, what would you want us to provide regarding Python2.7 support, considering Python2.7 itself will no longer be supported after 1st January 2020?

...

We're using RHEL7 within MOHC and while there will be no further development of central python2.7 installations from the end of 2019 we'll still have some operational forecasting systems working with this version until the migration completes in 2020.

As we've been a bit behind in publishing data sets for CMIP6 (the metric for success for me :-) ) I haven't been able to put resources into a python 2 to 3 migration and there are a couple of other library version jumps that we'll need to thoroughly test first (iris 1.13 to 2.x) and understand the performance implications of (iris 2 uses dask and is up to twice as slow as iris 1).

It may be that there isn't a specific need for us to pick up CMOR v3.6 any time soon (currently we use 3.4), but if there are changes that allow CMOR to produce new variables we would be unable to incorporate these changes until the second half of 2020 when we've migrated.

There are almost certainly ways to manage this. For example you could make CMOR v3.6 python3 only but continue with micro version updates to 3.5 to give the two equivalent functionality for a limited period. The duration of this overlap would then be a topic for negotiation between you and the user base.

[durack1]

@matthew-mizielinski thanks again, useful feedback that we can ponder as we ascertain resource availability for maintenance going forward

[durack1]

Just because this info is relevant to this thread

From: LC Hotline
Date: Monday, December 2, 2019 at 11:29 AM
Subject: LC Python 2 Support will officially end Python 2 support on January 1, 2020

Subject: LC Python 2 Support
 
The Python Software Foundation will officially end Python 2 support on January 1, 2020
(https://www.python.org/doc/sunset-python-2/). After that date the Python community
will not improve Python 2 even if a bug or security problem is found. However, Livermore
Computing will continue to support existing Python 2 installations on current systems
and will likely maintain a Python 2 installation beyond the January 1, 2020 on new
systems (and through major OS updates, e.g., TOSS 4) for the foreseeable future.
 
In the long term, users are strongly encouraged to port their Python scripts to Python 3,
as Python 3 will eventually become the default and because Python 2 will eventually
be retired from LC systems. We do not currently have an update or retirement date 
as such a move will be dependent on any major bugs, security issues, and existing
Python packages (e.g, numpy, scipy, etc.) support. Users will be given ample notice
prior to LC dropping support for Python 2.
 
More information about Python on LC systems can be found at
https://hpc.llnl.gov/software/development-environment-software/python. Below are
a few references that may help port Python 2 applications to Python 3:
 
https://docs.python.org/3/howto/pyporting.html
https://docs.python.org/2/library/2to3.html
 
Please contact the LC Hotline if you have any questions or concerns.

[sashakames]

It might be a challenge to get all sites to migrate their publisher installations to Python3 in a timely fashion. The work to migrate the code is in progress. So in order to keep the PrePARE integration current, which appears now to be crucial for CMIP6, I would be hesitant to cut off Python2 support too quickly -- at least for PrePARE as it is integrated with esg-publisher.

[mauzey1]

@sashakames @durack1
conda-forge will no longer support Python 2.7: https://github.com/conda-forge/cfep/blob/master/cfep-15.md

We could still keep Python 2.7 support for the source install and nightly build, but the stable builds from conda-forge will only be for Python 3.6, 3.7, and 3.8. I am currently working on getting Python 3.8 supported for the next release.

[durack1]

@mauzey1 wow, that’s a dramatic support move, what is their plan for py27 libraries, maintain with no future releases?

- We will provide an admin command that will add back Python 2.7 to any repo   on an opt-in basis.

For CMOR3 I suggest we take this path, keep current releases “floating” for py27, and don’t bother to add more steps to support this in future releases.

[durack1]

@matthew-mizielinski can you chime in here?

[mauzey1]

According to the conda-forge's announcements, you can still have Python 2.7 in your feedstock by adding an issue titled @conda-forge-admin add python 2.7 in the feedstock repo. I will try this. However, it will get rid of other Python builds so it needs to be merged into a separate branch.

[mauzey1]

@durack1 @taylor13 @sashakames @muryanto1

I am currently running into an issue with Python 2.7 regarding the testsrunner module from CDAT. A change was added to it that makes it incompatible with Python 2.7.

Traceback (most recent call last):
  File "run_tests.py", line 10, in <module>
    runner = TestRunnerBase(test_suite_name)
  File "/Users/distiller/project/macos_build/miniconda/envs/py/lib/python2.7/site-packages/testsrunner-v8.2_26_g3cf5a0b-py3.6.egg/testsrunner/TestRunnerBase.py", line 76, in __init__
AttributeError: 'module' object has no attribute 'get_start_method'

I have tried using older versions of testsrunner in the CMOR tests, but they were also causing errors. We could skip parts of the Python tests for 2.7 but I don't recommend this.

I think we have gotten to the point where we should discontinue support for Python 2.7 for CMOR. It has been over half a year since the end-of-life of Python 2, CMOR's conda-forge feedstock no longer builds for Python 2.7, and CDAT applications have also dropped Python 2.7 support.

[durack1]

@mauzey1 thanks so much for attempting to continue to support py2.7 with CMOR3. I had not realized how quickly anaconda/conda-forge etc would drop support for py2.7, which to be honest has shocked me.

I think planning forward, we'll need to drop py2.7 support, it's going to take too much time to continue to find workaround after workaround, so I think we've lost that battle.

For the preservation of the py2.7 versions that currently work, is it technically possible to pin this to existing versions of libraries that work, so that we can try and keep this limping along for a little longer? I think we'll need to note on the cmor webpage that py2.7 support is no longer provided, but for users that are willing to invest the time finding compatible libraries, the last supported version was 3.6.0-py2.7-pinned.

Does this sound like a reasonable conclusion to this issue, and not too much work?

[mauzey1]

@durack1
For now, you can install the Python 2.7 version of CMOR using conda create -d -n cmor_py27 -c conda-forge python=2.7 cmor to install the last stable version of CMOR that supported Python 2.7, CMOR 3.5.0. You can also install the latest nightly version using conda create -n cmor_nightly_py27 -c pcmdi/label/nightly -c conda-forge python=2.7 cmor=3.6.0.

As for the current issue with testsrunner, I think we could replace the command python run_tests.py -v2 -H -n1 Test/Test/test_python_CMIP6_CV_*.py with find Test -name "test_python_CMIP6_CV_*.py" -type f -exec python {} \;. That way tests won't require installing testsrunner. However, it won't collect the results of the tests in HTML files like testsrunner.

[durack1]

@mauzey1 I am happy for you to take a lead on how to best deal with this, however I am a little concerned that maintaining py2.7 support is going to be a never ending job, so am a little hesitant for us to commit much time to it