Python 2.7 deprecation timeline #1175
Comments
|
I have self-assigned this to take forward to PIs |
|
As we are now in Phase 1 I've disabled Jenkins MacOS builds for xia2 and regression builds for DIALS as described. I have removed links to Python 2 nightly builds from https://dials.github.io/installation.html#development-builds |
|
I have amended the timeline above following a request to delay Phase 2 until the end of June as to facilitate XFEL work. Since this is independent of xia2 I have split up Phase 2 into 2A (remaining at original date but for xia2 only) and 2B (end of June, everything else). Further, I've changed the wording to make clear that 'large scale refactoring' includes the removal of |
|
Having just taken an inventory of all our Jenkins build configurations I would just like to make a note that we will also reduce the coverage of testing at Diamond on
Python 3 test coverage of Obviously nothing changes in terms of code committed to |
||||||||||||||||||||||
If a test fails on Python 2 and is expected to do so then mark it with the 'python3' fixture. If a test fails during the pytest discovery/collection phase due to a SyntaxError in the test file, then add the filename to the exclusion list in conftest.py. If the SyntaxError happens due to an import in the test file then consider moving the import into the test function first.
Don't attempt to run Python 2 syntax/flake8 checks on newly added files. See dials/dials#1175 for the deprecation plan.
|
We are now in Phase 2A, and I have disabled xia2 Jenkins builds accordingly. Whereas xia2 is now free to move on, the policy for dxtbx/dials remains at "Python 3 allowed in new files and where Python 2 support is unreasonable or impossible" until the end of June. |
and make a warning more visible. cf. #450 and dials/dials#1175
|
This proposal from the DIALS East group was agreed by representatives of DIALS West. It is only fair that this is made public here
|
The check in the Azure pipeline is more powerful as it supports the Github Checks API. Leave the Python2 flake8 check until we disable that at the end of the month according to #1175.
|
We are now in Phase 2B, and I have disabled the DIALS MacOS Python 2 Jenkins builds and the MacOS dxtbx regression builds accordingly. Travis no longer runs Python 2.7 flake8 checks and regression tests. The policy for dxtbx/dials is now "Python-3-only code is allowed where it provides a demonstrable benefit over 2.7-compatible code." until mid-July. |
|
We are now in Phase 3, and I have disabled the relevant Jenkins tests accordingly. Travis no longer runs Python 2.7 syntax checks and tests on DIALS. Test fixtures will be removed shortly. The policy for dxtbx/dials is now "Python-3-only code is allowed where it would take additional effort to maintain Python 2.7-compatibility". Although we no longer test on Python 2.7 any patches to retain 2.7-compatibility are welcome until mid-September. |
Warning is only shown on first import, and can be silenced in Python 2.7 with
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore")
import dials
cf. #1175
|
I have disabled Python 2.7 syntax checking on cctbx/dxtbx on Jenkins due to cctbx/cctbx_project@9343e8a. |
- Remove references to python 2/3 cross-compatibility in advance of completion of dials#1175 - Greatly soften the request for clean branch history in pull requests. Since we aim to squash-merge most pulls now, this is much less important because it mostly won't be seen on main branch. - Pull requests should have `newsfragments` added to them
- Remove references to python 2/3 cross-compatibility in advance of completion of dials#1175 - Greatly soften the request for clean branch history in pull requests. Since we aim to squash-merge most pulls now, this is much less important because it mostly won't be seen on main branch. - Pull requests should have `newsfragments` added to them - Pre-commit is in many cases installed automatically now, so change the wording there. - Be less wordy in flake8 section
- Remove references to python 2/3 cross-compatibility in advance of completion of dials#1175 - Greatly soften the request for clean branch history in pull requests. Since we aim to squash-merge most pulls now, this is much less important because it mostly won't be seen on main branch. - Pull requests should have `newsfragments` added to them - Pre-commit is in many cases installed automatically now, so change the wording there. - Be less wordy in flake8 section - Describe isort
- Remove references to python 2/3 cross-compatibility in advance of completion of dials#1175 - Greatly soften the request for clean branch history in pull requests. Since we aim to squash-merge most pulls now, this is much less important because it mostly won't be seen on main branch. - Pull requests should have `newsfragments` added to them - Pre-commit is in many cases installed automatically now, so change the wording there. - Be less wordy in flake8 section
* Disable Python 2 testing (dials/dials#1175) * Clean up Travis build scripts * Retire Python 3 migration helpers
|
The 15th of September has finally arrived. It looks like the timeline was spot on, as some of the build job stages have started coming up with dependency package errors yesterday. I have disabled all Jenkins Python 2 testing for cctbx and dxtbx, disabled the dxtbx travis testing, and removed associated code crutches. We can now move on. |
- Remove references to python 2/3 cross-compatibility in advance of completion of #1175 - Greatly soften the request for clean branch history in pull requests. Since we aim to squash-merge most pulls now, this is much less important because it mostly won't be seen on main branch. - Pull requests should have `newsfragments` added to them - Pre-commit is in many cases installed automatically now, so change the wording there. - Be less wordy in flake8 section * Apply suggestions from code review Co-authored-by: Markus Gerstel <2102431+Anthchirp@users.noreply.github.com> Co-authored-by: David Waterman <dagewa@users.noreply.github.com>
As we are all aware Python 2.7 is now well past its use-by date.
DIALS will need to continue supporting Python 2.7 for a while as we still rely on this for a number of reasons (DLS operations, CCP4 bundling, Phenix bundling, ...)
This ticket outlines my proposal how we can support this critical need without obstructing ongoing development effort.
Executive summary
On March 15 we will start the release branch for DIALS 2.2. DIALS 2.2 will support both Python 2.7 and 3.6. Like DIALS 1.14, this will be a long-term-supported (LTS) release, which means we will provide support and builds, initially until the end of the year. This may be extended if there is ongoing need for supported builds.
Meanwhile we will reduce the Python 2.7 compatibility in the master branch in a phased manner over a period of 6 months. This is to avoid a scenario where massive changes are introduced that then make it difficult to backport code to the 2.2 release branch.
Phase 1 -- 15th of March 2020
We stop officially providing or linking to any 2.7 nightly builds
We aim to keep Python 2.7 compatibility in dials, dxtbx, and xia2
masterbranches and refrain from any large-scale refactoring to not artifically make maintaining the 2.2 LTS release difficult. Python-3-only code is allowed in new files and where retaining Python 2 support is unreasonable or impossible (multiprocessing case). Where this causes tests to fail we will set up a "skip-this-test-on-python-2" fixture.We will reduce the MacOS testing regime at Diamond to reduce the demand on our build machine.
Jenkins
Travis
Phase 2A -- 15th of May 2020
This is an intermediate step where only xia2 is affected. DIALS and dxtbx
masterremain in Phase 1 for 6 more weeks.We end all effort for Python 2.7 compatibility in xia2. Any large-scale refactoring, such as mass-removal of
future/sixstatements, should take the need for backporting fixes into the 3.0 release branch into account.With concurrent 2.2 and 3.0 release builds we will have increased demand on our build systems, so responses may be slower than usual.
Jenkins
Travis
Phase 2B -- 30th of June 2020
We still aim to keep Python 2.7 compatibility in DIALS and dxtbx
masterwhere possible. We aim to avoid large-scale refactoring, particularly in dxtbx. Python-3-only code is allowed where it provides a demonstrable benefit over 2.7-compatible code.With concurrent 2.2 and 3.0 release builds we will have increased demand on our build systems. We now reduce the testing regime, in particular we end the long-running regression tests for DIALS.
Jenkins
Travis
Phase 3 -- 15th of July 2020
We still aim to keep Python 2.7 compatibility in dxtbx
master. Python-3-only code is allowed where it would take additional effort to maintain Python 2.7-compatibility.We stop all 2.7 testing on DIALS and remove all test fixtures. However we bear Python 2.7 compatibility in mind, and patches to maintain Python 2.7 compatibility are welcome.
Jenkins
Travis
Phase 4 -- 15th of September 2020
We end all Python 2.7 compatibility effort.
Jenkins
Travis
The text was updated successfully, but these errors were encountered: