Python 2 and 3 compatibility #21

Closed
m45t3r opened this Issue Mar 30, 2014 · 8 comments

Comments

Projects
None yet
3 participants
Contributor

m45t3r commented Mar 30, 2014

Opening a new issue to discuss better about maintaing Python 2 and 3 compatibility. Some problems and possible solutions:

-Renamed modules: various modules was renamed on Python 3 (every module that started with Uppercase on Python 2 standard library was renamed to lowercase, and some modules like ttk was put on the group tkinter.ttk for example). This is relatively easy to fix using Try...Except ImportError, but maybe putting this on every module that needs it is not the cleanest way.

-Deleted built-ins: this is easy, we should only not use any of the following functions: apply, cmp, coerce, execfile, file, long, raw_input, reduce, reload, unicode, xrange, StandardError. Some of them were renamed (like raw_input was renamed to input), while others are not built-in anymore (reload is a function from imp module) and others are simply removed.

On the cases where the module was renamed we can do the following:

try: input = raw_input
except NameError: pass

-Print as a function: this is easy: just don't use print "string" anymore and instead use print("string"). To guarantee that no code uses the old type of print you can include:

from __future__ import print_function

-Divisions: Python 2 returned a int number on a division if both operands are int too (like 3/4 was 0 and not 0.75 like expected). Python 3 always returns the float, and introduces a new // if you want int division (this works on Python 2 too). To garantee that no code uses the old division you can include:

from __future__ import division

-Absolute imports: Python 3 doesn't allow implicit relative imports anymore. Every relative import should be explicit:

package/
    __init__.py
    subpackage1/
        __init__.py
        moduleX.py
        moduleY.py
    subpackage2/
        __init__.py
        moduleZ.py
    moduleA.py

On this example I can't include moduleY from moduleX using:

import moduleY

It should use the following:

from subpackage1 import moduleY

To get the same behavior on Python 2:

from __future__ import absolute_imports

-Unicode literals: This is the most important and most trick one. Python 3 use unicode literals while Python 2 use strings. So we should explicit use unicode strings so we have the same behavior on Python 2 and 3. We can convert every literal to unicode using:

from __future__ import unicode_literals

The problem is, some Python 2 modules expect str instead of unicode. And some things is broken with above statement on Python 2 (e.g. unicode docstrings). See http://python-future.org/imports.html#unicode-literals for details.

The second option is to explicit mark each string with unicode, e.g:

print(u'Hi, I am a unicode string!')

But this should only work on Python 3.3+ (http://docs.python.org/3/whatsnew/3.3.html).

Things to discuss:

-Minimum supported version for both Python 2 and Python 3. We already support only Python 2.7 because of ttk and argparse (argparse is available for older versions as a PyPi module, ttk I think not), but maybe there is some interesting on supporting older versions of Python. Just remember, while it's relatively easy to support Python 3 compatibility with Python 2.6/2.7, going to lower versions is a PITA (no __future__ statements for example).

For Python 3 itself, the lowest version that we supports now is probably Python 3.2 (don't take my word at it, I only tested shim with Python 3.3), since we use argparse module that was introduced on this version. But argparse exists on PyPi, so in theory we can support Python 3.1 too, but I don't think it's worth it.

For more info about maintaing Python compatibility between major versions, see http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/.

-Automated tests. Ideally we should have automated tests to every major Python version we support (i.e. Python 2.7/3.2/3.3/3.4).

-To use or not to use __future__ statements. Excluding the unicode problem, every other __future__ import are only put to garantee that no one writing code/testing on Python 2 uses incompatible versions of the code, since these funtionality exists on Python 2. The recommended would to put the following import on every file:

from __future__ import absolute_import, division, print_function

And maybe unicode_literals too if we do use it.

-Another option would be to use six (https://pythonhosted.org/six/) module that takes care of the majority of the problems, but if we only need to support relatively new versions of Python 2 and 3 we should not needed it (six is recommended if you do need to support ancient versions of Python like 2.5 while maintaing compatibility with Python 3).

python-future is another module that does the same, but aparently has a better documentation (http://python-future.org/quickstart.html). Since the code is new, we could simple add the following to every module:

from __future__ import (absolute_import, division,
                        print_function, unicode_literals)
from future.builtins import *

And starting writing Python 3 code from now.

-Yet another option would be drop Python 2 or Python 3 (please not) support so we don't need to deal with these problems.

Some nice links:
-http://docs.python.org/3/howto/pyporting.html
-http://python3porting.com/

Contributor

m45t3r commented Mar 30, 2014

IMHO:
-Even python-future doesn't support Python <=3.2 so we should stick with Python >=3.3.
-For Python 2.x series we should support only Python 2.7, that is 4 years old.
-To make things simple, we should't use six (it does too many things), but maybe python-future would be nice. It's a very nice library btw.
-If we need to drop a major version, I would prefer to drop Python 2. It simply too old and all Python development is happening on Python 3 (for example, new modules to standard library and language features).

Owner

swong15 commented Mar 30, 2014

Thanks for the writeup.

The only reason I'd be hesitant to drop python 2 completely would be because Macs come default loaded with python 2.7.

However, I think dropping Python 2 support would be the best option.

It seems like the programming overhead associated with what should eventually (and potentially already is) a completely obsolete version of Python far outweighs the benefits of maintaining support for people using 2.7. For us it's potentially lots of headbanging and messier looking code and for users its the time saved installing python 3 (which is extremely easy). I'm all for simplicity and I really don't see the point in sacrificing that in order to make installation one command shorter.

I was a little hesitant because Macs come loaded default with python 2.7 and being stuck in my little college bubble I assumed 2.7 would be the safer option to go with. Unless theres some super compelling reason to keep Python 2 support, I'd be completely fine with just scrapping Python 2 support if everyones fine with it.

tony commented Mar 30, 2014

https://github.com/kaaedit/kaa is python 3 only and its' codebase is awesome.

If you need some advice on python 2.7 compatibility I can be of assistance. But I think sticking with python 3 is sensible, many linux distros have it. No modern computer system should have a problem with a system package of python 2 and 3 side by side.

Regarding the mac issue, https://github.com/Homebrew/homebrew/wiki/Homebrew-and-Python. Out of the box AFAIK - python 3 may not exist, but homebrew makes a cakewalk of it. I just had both set up last week no problem.

tony commented Mar 30, 2014

@m45t3r : I think the most important thing you mention is http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/. Ronacher's _compat libs do the trick well. I just helped out at monetizeio/sqlalchemy-orm-tree#20 and implemented a compat module derived from werkzeug/jinja2/flask.

Contributor

m45t3r commented Mar 30, 2014

If you want to support only Python 3, I completely support this choice. Python 3 code is cleaner, more consistent and faster (or so some people say). The Python 3 standard library is gaining more features and some things is better too.

Contributor

m45t3r commented Mar 30, 2014

If we do choice to support Python 2 and 3 I think using the python-future would be the easiest choice, using something like:

from __future__ import (absolute_import, division,
                        print_function, unicode_literals)
from future.builtins import *

But the user would need to install python-future, and if the user needs to install something maybe it simple easier to say "use Python 3".

Contributor

m45t3r commented Mar 30, 2014

If we decide to remove Python 2 support I can do it. It's easy to do, just remove some imports.

Owner

swong15 commented Mar 30, 2014

Let's drop python 2 then. Go ahead and start ripping out the imports

m45t3r closed this Mar 30, 2014

m45t3r referenced this issue in anishathalye/dotfiles_template Jun 15, 2014

Closed

Check Python binary version #2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment