when should we drop support for legacy python 2?

[anthrotype]

Python 3.6 is coming out in a couple of weeks with loads of new cool features.
Alas, we are still stuck with supporting legacy Python 2.7.

I think we need to start thinking about until when we would like to keep support for legacy Python.

We can't keep stretching the code -- as well as ourselves -- for long, trying to maintain compatibility with such diverse languages.

The main issue we face is that font editors (mostly OSX based) that require fonttools and sister libraries, have so far contented themselves to link their apps with the Apple-provided Python 2.7 framework. Of course, we can't know when exactly Apple will finally decide to ship a modern Python with macOS.

However, it's not difficult to bundle a Python distribution along with the font editors themselves (e.g., I found a blog post explaining how to do that, and doesn't sound that complicated). I'm also willing to lend my spare time to help font editors developers making this happen.

So, I propose we set the end-of-support date one year from now, on 12 December 2017.

WDYT?

[anthrotype]

BTW, did you know that FontTools will turn 18 on the 12 December 2017?
(at least according to Github, but @justvanrossum can confirm).

[justvanrossum]

I think Python 2.7 will be around for a long time, but I see your point, too. In the UFO world there is a lot of Python 2.7. There's a chain of dependencies there, that has vanilla at the bottom. I think most of us wouldn't mind so much moving to Python 3, but we need to see how to organize efforts to actually do it. As you know, it's not trivial. And still, while there are many nice things about Python 3, the benefits of switching are relatively small compared to the costs.

I think it's unlikely that a big application such as RoboFont will have switched to Python 3 in one year from now, so I'm not sure how realistic (or: "friendly") your deadline will turn out to be.

[anthrotype]

Thanks Just. I just wanted to set the ball rolling :)
Let's at start talking about how to make this happen and what the obstacles are.

As for embedding python instead of using the legacy system framework, that can be done relatively easily.

You mentioned Vanilla as a library which needs to be ported to py3. What are the other ones? I can help futurize them.

What else?

[justvanrossum]

Maybe the rest of the hesitation is more about non-library code or private library code. There's a lot of stuff, that is used a lot but is not publically visible. RoboFont, MetricsMachine, Superpolator, Prepolator.

Then there's the issue for RF regarding extensions, and people's private code. It's hard to estimate what impact a switch to Py3 will have for the RF community as a whole.

(I'm not worried about embedding at all. Depending on the /System Python has had it's fair share of disadvantages.)

[typemytype]

I dont mind the little changes required to make packages work on py3 (most of the work would be tupling coordinates)

Im worried most about building apps.
So I would like to propose to set the start of that year from the moment you can provide an example 'hello world' app build with your from your blog post without py2app with vanilla that runs on 10.9+. Or even a working DrawBot, as an extended example.

[anthrotype]

Challenge taken! 👍

[justvanrossum]

DrawBot would be an excellent start indeed.

[anthrotype]

Ok, I'll play with DrawBot.
Btw, py2app seems to support python 3, at least from a quick reading of the docs. I'll let you know how it goes soon.

[typemytype]

Additional requirements:

• No WX
• must be codesignable

There is a py3 version of pyObjC

Good luck!

[typemytype]

Maybe start with https://github.com/typesupply/vanilla/tree/master/Demos/SimpleApp ;)

DrawBot is also not such a complex app, just had more requirements.

[anthrotype]

thanks Frederik!

[anthrotype]

pyobjc compiles just fine with python3, however I'm having issues with py2app.

The latter does not seem very actively maintained, the last commit was more than a year ago:
https://bitbucket.org/ronaldoussoren/py2app/commits/branch/default

I think I'll give pyinstaller a try. This one is not mac specific, but supposedly works the same for all three platforms.

@adrientetar has used it for Trufont, right?

[adrientetar]

@adrientetar has used it for Trufont, right?

I previously did, yes.

[davelab6]

What do you use now?

…

On Dec 11, 2016 1:22 PM, "Adrien Tétar" ***@***.***> wrote: @adrientetar <https://github.com/adrientetar> has used it for Trufont, right? I previously did, yes. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#765 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAP9yyJW48AuwBaHXUbFyFwRWKFI6lweks5rHD95gaJpZM4LJ5mk> .

[adrientetar]

What do you use now?

pyqtdeploy

[typemytype]

an observation for the other end of the scripting experience axis:

Non script-ers, who fe would like to compare font data with xml, are forced to install py3, that seems like a big and dramatic requirement...

[anthrotype]

well, it's not a big deal to run an installer from a dmg, is it?
https://www.python.org/downloads/

[benkiel]

@anthrotype I'm just wondering, what does Python 3.6 add that outweighs the issues with dropping 2.7 support?

[davelab6]

On 12 December 2016 at 05:01, Ben Kiel ***@***.***> wrote: @anthrotype <https://github.com/anthrotype> I'm just wondering, what does Python 3.6 add that outweighs the issues with dropping 2.7 support?

The Unicode and division improvements seem important to me for typical font scripting :)

[benkiel]

@davelab6 I understand what Python 3 adds, I was referencing "Python 3.6 is coming out in a couple of weeks with loads of new cool features."

[moyogo]

The general idea is that Python 3 is actively being developed, with improvements and new features, while Python 2.7 only gets bug fixes.

[behdad]

I think it's unlikely that a big application such as RoboFont will have switched to Python 3 in one year from now, so I'm not sure how realistic (or: "friendly") your deadline will turn out to be.

Even if ten of us do a weeklong hackathon in the Netherlands? ;)

[behdad]

At Google, we just moved from using Jython to Python 2.7 for running subsetter. So, from our point of view as well, we need this probably longer than a year. Jython workarounds can go though as far as we are concerned. Thanks! cc @garretrieger

[davelab6]

Ben, f-strings and underscore literals seem like cool features that will also be popular in the font scripting community

[anthrotype]

@benkiel here's a good video from Brett Cannon, a python core dev, explaining what's new in python 3.6 https://youtu.be/hk85RUtQsBI

[behdad]

@benkiel here's a good video from Brett Cannon, a python core dev, explaining what's new in python 3.6 https://youtu.be/hk85RUtQsBI

Heh. Brett and I used to drive to Google Waterloo together.

[rsheeter]

Long time since the last update. Any idea when this might start to move? - helpful for Google Fonts planning.

Out of curiosity, are there specific things in Python 3 that are desirable for FontTools?

[anthrotype]

hey Roderick, glad you ask

any idea when this might start to move?

well, if it was only up to me, the earliest the better, e.g. 1st of January 2019 (about in six months time) could be my most optimistic date.

are there specific things in Python 3 that are desirable for FontTools

I could list many things, but I'm too lazy. here's a good summary
https://snarky.ca/how-to-pitch-python-3-to-management/

I'm not joking: I spend 30-40 % of my dev time trying to make the same code base work reliably on both pytho2.7 and python3.x. There are so many hacks, compromises and workarounds that we could ditch if we weren't supporting legacy python. Plus, we could use lots of new cool features only available on modern python, and it would also be faster (3.7, especially).

Behdad started working towards this in 2013 with his fonttools fork. Five years later, RoboFont.app (another major client of fonttools) is now python3.6 only.

The argument that "the macOS built-in python is still 2.7" is not a good one. Apple is not interested in updating the built-in python to 3, because the reason that exists is to allow running some other system utilities written in python, and is not meant to be used (or extended) by macos users.
Mac native applications can and should embed the Python3 framework instead of relying on the outdated system one. Users can easily install python3 with homebrew or by downloading the official installers from python.org (windows users have been doing that since ever). Linux users can as easily do ./configure && make install from Python source, but it's likely their distro package manager already supports some modern version of python3.

If we ditch python2.7, probably the new baseline python should become python3.5 at least for another year or so (e.g. Ubuntu Xenial 16.04 LTS from 2 years ago still ships with python3.5) -- although I'm really tempted to require python3.6, so we can use f-strings and type annotations for variables (not just for function args).

Anyway.. we got to move on. I am willing to help Google Fonts team to help them make this happen.

[chrissimpkins]

There are so many hacks, compromises and workarounds that we could ditch if we weren't supporting legacy python. Plus, we could use lots of new cool features only available on modern python, and it would also be faster (3.7, especially).

There is a good argument that projects that continue to do such things simply act as enablers for the ongoing existence of the Python 2 interpreter. Everyone wants to kick the code base transition can down the road because it is time-intensive unproductive work but at some point the entire Python development community needs to simply cut the cord or Py2 will never go away. Those who find that they have a tremendous amount of work to do in response to upstream transitions here have ignored the writing on the wall that has been present for years (very literally in this case, this issue was filed in 2016). Generally speaking, it seems to be an issue of everyone in the Python dev community waiting for someone else to make the first move and we have seen how well that has worked to make a timely transition occur. Py2 EOL seems to be/should be the impetus that everyone needs.

[chrissimpkins]

Great comment on SO about the Py2 sunset... :)

It's like the Windows XP of the programming world...

[jenskutilek]

I finally took the plunge and installed Python 3 on my main macOS 10.12 machine. I’m glad to report that I found no problems so far using it for font-related tasks. It doesn’t interfere at all with the system 2.7 Python that is still needed for some stuff (FontLab 5, anyone?). But that’s just one more site-packages folder to keep organized ;)

[anthrotype]

https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7

it was a large gain of time to be able to only develop for a single version of python. Avoiding the use of conditional imports, being able to rely on type annotations, and make use of the newly available Python APIs were some of the advantages that made us more productive. Especially as most of the work on IPython is done by volunteers who work on nights and weekends, with only a couple of minutes here and there, this often made the difference between a patch reaching completion, or the contributor moving on to other pastures.

[anthrotype]

I propose 1st of January 2019 as the date when we stop supporting legacy python.
We could still try to maintain a bugfix only python2 branch another six months after that, but starting the new year all new features will only be for python3.

[twardoch]

I’m still amazed how 10 years after the release of Python 3.0, the Python community did not dare to reverse the stupid "print" decision that, I believe, contributed to slowing down the overall Py2->Py3 wide migration like three times. It's literally the biggest reason — it's like: you keep 100 million trivial scripts working or you make 99 million of them stop overnight, and cause frustration to millions of very casual users

[anthrotype]

Friendly reminder: in about two months from now, fonttools will drop support for python2.7. The master branch will fork and a py27 branch will be maintained for another six months as bug-fix only; all new features will be for python3.5+.
Reach out if you need assistance in this transition.

[medicalwei]

I also notified in Debian's mailing list that upstream is dropping Python 2 support:
https://lists.debian.org/debian-fonts/2018/11/msg00000.html

If they don't migrate these fonts listed in the mailing list above will be unable to build, then eventually removed from Debian repositories.

Notifying nototools is also needed, otherwise noto fonts won't be able to build as well:
notofonts/nototools#472

[anthrotype]

nototools is not required to build the Noto fonts, fontmake is. And one can always pin the exact version of fonttools, the current versions will continue to work and be available for install on python2.7 indefinitely. It’s only the new versions from 1 Jan 2019 onwards that will be py3 only.

[anthrotype]

The end of the year is approaching. Time flies...
I hope that we still all agree about dropping support for python 2.7 from beginning of next year.
The plan is do it gradually. For the first few months we can commit to backporting bugfixes to the py27 branch and do patch releases (e.g. 3.34.1, 3.34.2, etc.).
At the same time, in the master branch we start dropping py23 compatibility code and add new features only in there.
If for any reasons you still are clinging to legacy python, now is the time to speak up and raise your concerns. Oh well, a few months ago would have been better, but it's never too late to discuss these things.

[justvanrossum]

I think it's important that we communicate clearly that fonttools won't disappear from 2.7, but that new features will only be added to the 3.x version.

[anthrotype]

Of course, we shall write that in the README and in the releases notes in a prominent position.
The trick is to use python_requires=">=3.5" keyword in the setup.py script, for all the new versions of fonttools 4.0. Then we can upload both fonttools 3.34.x and fonttools 4.x on PyPI, and pip will automatically download the most recent version that is compatible with the current python.
This is how, e.g., ipython project does it for example. If you pip install ipython on python2.7 you get some 5.x version, whereas if you do it from python3.7 you'll get the latest one.

[anthrotype]

I just added a note to the README saying that we plan to only do bugfix releases for python 2.7 in the next six months or so, and that all new features starting from 1st of January 2019 (next month) will be python3 only.
4a15333

[tshinnic]

Hmm, came across an old issue #234 and wondered: is there a separate issue for "cleanup source assuming python 3 environment/capabilities"?

Looking at that old issue, I notice there's still cruft around like newline='\r', where - if I understand correctly - the newline= and sep= could just be removed? As it is the code is of two minds.

[anthrotype]

Thanks for the heads up. We haven’t started the transition to py3-only source yet (I’ve been busy with other work related things these weeks). Anyway, the issue about “\r” line endings in some old part of the codebase that you linked to is orthogonal to the switch to py3 and can be tackled independently of that. But very low priority. PRs always welcome.

[khaledhosny]

I suggest we mark the next release as the last Python 2 (or perhaps Python < 3.6) compatible one. After the release drop CI support for building with unsupported Python versions and gradually remove any code or hacks for said versions of Python. We can’t keep dragging our feet forever.

[anthrotype]

I'm going to cut a new release today, which 🤞 will be the final python2.7-compatible release of fonttools.

[anthrotype]

By Python 3, I mean >= 3.6. If we had dropped support a few months ago, I would have included 3.5, but I think by now we are in a position to only support truly modern python.

[anthrotype]

I pushed a py27 branch here on the upstream repository, a snapshot of latest master immediately after 3.44.0 release. Just in case we want to backport any critical bug fixes to the fonttools v3.x series.

[anthrotype]

@justvanrossum Are we ready to cut the first fonttools v4.0.0 release?
I'd like to do it today, as I'll be on holidays from tomorrow, and then attend ATypI Japan.

[justvanrossum]

Go for it! Thanks.

[anthrotype]

Done https://github.com/fonttools/fonttools/releases/tag/4.0.0

We can finally close this :)

[anthrotype]

LOL.. Github is partly down and doesn't let me close the issue..

[anthrotype]

I had to delete and re-push the 4.0.0 tag to trigger a new Travis CI build, as the previous one failed because Github was down. It should be up on PyPI any minutes now
https://pypi.org/project/fonttools/4.0.0/