Switch to py3

[heikoheiko]

Q: Shall we also break py2 compatibility and use new py3 features?
Yes, py3 only.

General information about the transition can be found in

• https://docs.python.org/3/howto/pyporting.html
• http://python3porting.com/preparing.html

Preparation

• replace encode('hex') with a py3 compatible variant (More py3 fixes #1158)
• replace decode('hex') with a py3 compatible variant (More py3 fixes #1158)
• use compatible imports
• get rid of own dependency forks
  • pystun
    • looks like this is not maintained any more, what do we do about it? Seems to work for now.
  • ethereum
    • We'll stay with 1.6.1 for now. It's compatible with py3 and doesn't break the API. In the long term we want to move to web3.py and py-evm.
• Check library dependencies, caniusepython3 lists 5 dependencies that need to be updated:
  • filelock
  • gevent-websocket
  • marshmallow-polyfield
  • miniupnpc
  • tinyrpc

Switch

• Remove usage of [value|keys|items]iter
• Remove future imports
• Remove __future__ imports
• bytearray objects work differently

Fix tests

This is an overview over which tests have been fixed already. See the palango/py3-fixes branch

• unit
• smart_contracts
• property
• long_running
• integration
• api

Once on python3

• Check division operators

• Remove python 2 variants

• See where list(...) around old iter calls is needed

• switch HTTP statuses to https://docs.python.org/3.6/library/http.html HTTPStatus

• Add type annotations to important interfaces

• Run mypy analysis, see where type signatures make sense

  • see follow up: Add mypy analysis to CI #1206

• Adapt linux and macOS build infrastructure for py3

  • see follow up: Adapt linux and macOS deployment infrastructure for py3 #1205

py3 porting gotchas

• Blockchain calls that used to return None now return b''
• Blockchain calls that used to return str now return bytes, we have to define how we want to handle this

[LefterisJP]

@heikoheiko Please let's not break py2 compatibility.

But yes we should make raiden py3 compatible and have a separate tox environment for py3 just like we have in pyethereum.

[andrevmatos]

I'm going to work on this for next sprint. As most of our dependencies already announced they're deprecating py2 support, I'd like to work in the direction of full migration to python3 when this gets fully solved.

[LefterisJP]

As discussed in the sprint planning meeting I have completely changed my mind since last year and I am wondering why do we want to keep py2 compatibiity instead of just deprecating it?

@konradkonrad @heikoheiko @hackaugusto Are there any reasons to support both py2 and py3 instead of just opting for a complete switch to py3?

The way I see it having a hybrid approach we end up putting quite a bit more work on ourselves as we have to write hybrid code for both python versions.