Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Best practices for making code compatible with Python2 and Python3 #2184
Yesterday we have been discussing (see #2181) if using six is a good option for us and there seems there is consensus about it.
We are investigating ways to automate the code porting and we found out the tool sixer.
We now want to write a script to automatically run sixer on our packages.
Anybody who has suggestions (or is aware of some reasons to not use this approach) is invited to discuss on this issue.
My personal opinion: using sixer sound a really good idea
Although I guess, I have barely tried six before, that six is mostly safe, our test matrix is quite complex enough to give it one commit on each package and then have fun trying to figure out which one is responsible of the breakage.
I love this talk from this year's pycon: https://www.youtube.com/watch?v=66XoCk79kjM what relates this talk to this issue is: "one commit one specific change", i.e. don't fix 300 different little changes on a single commit, because even if 300 little changes one on their own are really small and innocent, 300 at the same time are an easy way to slip simple mistakes that go unseen, because, who reviews a boring pull request doing only 300 small cleanups that spans 90 files with a diff of +1000 lines? If that same diff is split into 300 chunks is far more easier to spot where the mistake has been made (if any).
I started to document the process. Feel free to update as needed.
Porting Plone packages to Python 3
Porting eggs with six, sixer and python-modernize
Install sixer and modernize
Migrate a package using sixer
Pick a package to work on. Add it to #2368 with your name. If the package is not yet in
Run sixer for a complete package in coredev to get an idea how much will need to be changed:
Create a new branch for the package you are working on:
Make all changes in-place:
Afterwards run isort with the Plone settings. If the settings are configured globally and isort is in the path, then sixer can be run like this instead::
If that does not work (for some it does not) you have to check that sixer did not
Now you need to:
Migrate a package using modernize
This runs all fixers except one that adds
Same as with sixer you need to check each change for its necessity and sanity!!!
Wrapping up the changes on a package
If the tests pass in python 2.7 and you think you did your best to prepare the code to be able to at least start up in python 3 do the following:
Running Plone on Python 3
Plone does not yet run or even startup in Python 3 but trying it is a nice way to find and fix py3 issues without guessing.
The config depends on
It also removes a lot of parts that either break at the moment or are not
Clone coredev and use branch 5.2::
Create py3 virtualenv::
Startup will fail since a lot of code is not python3-compatible yet.
Frequent errors include:
changed the title from
Best practices for making code compatible with Python2 and Python3
Document best practices for making code compatible with Python2 and Python3
Jan 22, 2018
@pbauer thanks for putting some documentation!!
May I ask one question? According to wikipedia by 2020, only Python 3.6 will still be supported (ok, 3.5 as well, but only until September).
Does it make sense to support 3.4 at all? Given that our testing setup is complex enough, having to run all the tests against 2.7 and 3.6 will be already a lot of time to wait to merge pull requests, the more python versions we try to support the longer the whole process it will be...
@gforcada good question. I simply thought we would support the same version as Zope. But it makes sense to only support 3.5. and 3.6 since 3.4 reaches end of live in march next year.
Some things that sixer does not pick up:
This was referenced
Mar 19, 2018
As an addendum to @pbauer 's instructions to get coredev-py3 running:
I ran into problems running
Installing it manually via
This means that I had to install the Python3 development package (which on my system is done with
Maybe this helps someone.
I think that unicode could have a full blown article in your porting code guidelines.
I tried Plone Python 3 and searching for news, one of the first thing I tried, is crashing.
The Python error message is difficult to understand, path[len(site_path) + 1:] is a bytes object, that's the whole problem, if it was a str it would work.
I grepped the python code of the git-cloned coredev for encode('utf-8') and got about 250 hits.
Not that all these hundred of hits will be bad of course. But they could. And it may not even be especially obvious. So a guideline on Unicode and porting Plone to Python 3 could be good to have, also for add-on writers who could write not compatible code - even right now
@ale-rt : yes it seems to work on both Plone for Python 3 and Python 5.1.2.