For Sage 9.2: Remove python 2 support from sagelib

[embray]

Python 3 support (#15530) was completed for Sage 9.0 (thanks in particular to the leadership of Frédéric Chapoton). Official support for Python 2.7 by the CPython core developers is ending January 1, 2020.

To enable major package upgrades and other changes that require Python 3.x (#29141),
this ticket removes Python 2 support in Sage for Sage 9.2.

Depends on #26403
Depends on #28660

CC: @timokau @fchapoton @dimpase

Component: python3

Keywords: days101

Author: Erik Bray

Branch: f368e32

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/28000

[jhpalmieri]

comment:3

What needs to be done for this ticket?

[embray]

comment:4

Go through and remove all hacks related to supporting Python 2. It will be interesting to see how big a change it takes even now. It's rather early to go through and do that in earnest, but it would be fun to just try and see what it takes.

[slel]

Changed keywords from none to days101

[jhpalmieri]

comment:6

One example: #27696 (remove the doctesting hack for "long", which just converts it to "int" when using Python 3.

[slel]

comment:7

Also, this ticket can be set as a dependency for other tickets that depend on the
removal of Python 2 support. Many dependency upgrades will depend on that.

Might it make sense to migrate the whole Jupyter part to Python3-only versions
before completely removing Python 3 support? Even when building Sage for
Python 2, we ship Python 3 too anyway.

[embray]

comment:8

I made some progress at days101 removing all dependency on the six module for Python 2/3 support and that alone was a HUGE change (and not even quite finished yet). Will post a branch once I'm done with that.

[embray]

comment:9

Here's some of my progress on this so far: Probably one of the biggest, most pervasive steps, being complete removal of dependency on six.

Of course, this step is optional--we can keep using six while otherwise dropping support for Python 2. But this makes it quite explicit.

It's pretty hard to avoid making a patch bomb of this. It's not all bad though--I started this months ago, and then once I got it finished I rebased on current develop and only had about a dozen minor merge conflicts.

[embray]

Branch: u/embray/python3/ticket-28000

[embray]

Author: Erik Bray

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

8f3cacb

Trac #28000: Remove sagelib dependency on six

[sagetrac-git]

Commit: 8f3cacb

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

736c7a2

One Python 2-specific block that did not use six.PY2.

[sagetrac-git]

Changed commit from 8f3cacb to 736c7a2

[jhpalmieri]

comment:12

Other tasks:

• remove or revise the doctests labeled # py2. Ideally, all such tests should be evaluated on a case-by-case basis, some should just be removed, but perhaps some should be modified to work with Python 3.
• With Python 3 build, do not install python2 #28426: with a Python 3 build, do not build Python 2.
• remove some packages which are only necessary with Python 2: typing (py3: typing should not be installed in Python 3 #28499) is the only one I know of currently, but I wouldn't be surprised if there were others.
• remove the legacy Sage notebook — at least, I don't think it works with Python 3 now, and I don't think anyone is working on it. This means we should deprecate it ASAP. (Make the sage notebook optional #25837)

[embray]

comment:13

My branch for this has some other in-progress changes that it seems I haven't pushed yet. I'll do so next time I rebase this branch.

[embray]

comment:14

Oh right, one of the changes I started was to remove unnecessary from __future__ imports, but it seems until we have #26403, to my slight surprise, some of them are still needed in Cython modules, e.g. to have the print() function by default.

[embray]

Dependencies: #26403

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

d2c4427	Trac #28000: Remove sagelib dependency on six
aab5d3b	One Python 2-specific block that did not use six.PY2.
50b8951	Trac #28000: Get rid of most `__nonzero__` references and/or replace them with __bool__

[sagetrac-git]

Changed commit from 736c7a2 to 50b8951

[embray]

comment:17

Rebased, but no guarantee there are no new bugs or new uses of six I didn't miss. I didn't check yet.

[sagetrac-git]

Changed commit from 50b8951 to 1f6a41d

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

972f431	Trac #28000: Remove sagelib dependency on six
4c35164	One Python 2-specific block that did not use six.PY2.
1f6a41d	Trac #28000: Get rid of most `__nonzero__` references and/or replace them with __bool__

[embray]

comment:19

Updated again after removing two new uses of six that have since appeared.

[slel]

Changed dependencies from #26403 to #26403, #28660

[jhpalmieri]

comment:21

Do we remove all doctests tagged # py2 here or on a separate ticket?

[embray]

comment:22

You could do it here, or a separate ticket (and make it a dependency of this ticket). It's going to be a patch bomb no matter what, but I don't see the harm in trying to split it up wehre possible.

[mkoeppe]

comment:27

Needs rebasing!

[jhpalmieri]

comment:29

Related: #27826

[jhpalmieri]

Changed branch from u/embray/python3/ticket-28000 to u/jhpalmieri/python3/ticket-28000

[jhpalmieri]

comment:31

Rebased. I haven't seen activity from embray for a while. Perhaps this is ready for review?

---

New commits:

f368e32

trac 28000: rebase commits from Erik M. Bray :

[jhpalmieri]

Changed commit from 0e37a95 to f368e32

[mkoeppe]

comment:32

Let's see if this finds a reviewer

[dimpase]

Reviewer: Dima Pasechnik

[dimpase]

comment:33

testing.

[dimpase]

comment:34

OK, this makes sense and appears to work. A long, but straightforward patch, also touching quite a bit of whitespace (which probably could have waited for another place, but OK).

Should removing Python2 spkg be done elsewhere?

[mkoeppe]

comment:35

Replying to @dimpase:

Should removing Python2 spkg be done elsewhere?

Yes, see #29141 (Meta-ticket: Remove Python 2 support for Sage 9.2; upgrades enabled by the removal)

[vbraun]

Changed branch from u/jhpalmieri/python3/ticket-28000 to f368e32

[timokau]

Changed commit from f368e32 to none

[timokau]

comment:37

＼(＾O＾)／

Hopefully the divergence between sage-the-distribution and sage-on-distro will decrease now and packaging will become simpler again.

[embray]

comment:38

Thanks for finally getting this ticket merged! Sorry I haven't had much time to work on it. As others have noted this does not take care of all tasks related to dropping Python 2 support--it was just me doing as much as I could in one place. Took hours!

[vbraun]

comment:39

This breaks on Python 3.5, I've created #29753 for that

[jhpalmieri]

comment:40

Does anyone know of any docker setups from tox.ini that have Python 3.5?

[dimpase]

comment:41

Python 3.5 will be gone in Sept, why would wr spend time trying to support it?

[mwageringel]

comment:42

This accidentally broke the Macaulay2 interface. Follow up at #29759 with a trivial fix.