Skip to content

Latest commit

 

History

History
80 lines (60 loc) · 4.18 KB

python3.md

File metadata and controls

80 lines (60 loc) · 4.18 KB
Raw

Python 3

Changes

  • Utilize from __future__ imports to make code compatible with multiple python versions
  • Find/change old dependencies -> use dependencies that are backwards compatible
  • Improve tests to catch changes made using the 2to3 script
  • Run 2to3 script, fix failing tests, remove deprecated code

Notes on changes I've made and what to be aware of:

Currently, the repository is compatible with both python 2.7 and 3.5+ through the use of from __future__ statements, which brings python 3 style printing, importing, unicode literals, and division into python 2.7

If the code is being ported entirely to python 3 (or higher), these from __future statements can be removed However, there are some deprecations between python2.7 and 3 that need to be addressed For example, running the command ./lint.py deprecated-method will show that there is currently one method (get_arg_spec, in formatter.py) that is set to be deprecated. However, the function doesn't exist in python 3.5+, but its replacement doesn't exist in 2.7! So, if updating entirely to 3.5+, this can be fixed (by using the newer function, inspect.signature).

Running the full test suite will also indicate that some behaviors have been deprecated with other modules. For example, numpy arrays cannot be accessed by a floating point number, and the old style of opening files without a context manager will generally result in a warning. These can be silenced, but It's probably best to change them!

Pickling

Some files generated by the pickle module are not backwards compatable. If switching entirely to python 3.5+, pickle files from 2.7 can be loaded, but new files (generated with the latest protocol in python 3), wont work with python 2, unless a different protocol is specified. (However, specifying different protocols everywhere could become tedious)

Some examples:

"Normal" pickling in python 2

Python 2.7.12 (default, Jul  1 2016, 15:12:24) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> data = 'GxGxGy'
>>> import pickle
>>> with open('2.pkl', 'w') as picklefile:
...     pickle.dump(data, picklefile)
>>> with open('2.pkl', 'rb') as picklefile: # Notice that python2 pickle doesn't care if the file is in str/bytes
...     print(pickle.load(picklefile))
GxGxGy

Python3 requires wb/rb, and introduces a new protocol

(It also uses cPickle by default)

Python 3.5.2 (default, Jul  5 2016, 12:43:10) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle # Which is cPickle by default, with some new protocols
>>> data = 'GxGxGy'
>>> with open('3.pkl', 'w') as picklefile: # Pickle uses wb/rb in python 3+
...     pickle.dump(data, picklefile)
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
TypeError: write() argument must be str, not bytes
>>> with open('3.pkl', 'wb') as picklefile: 
...     pickle.dump(data, picklefile)

>>> with open('2.pkl', 'rb') as picklefile: # Old files can be opened
...     print(pickle.load(picklefile))
GxGxGy

Files pickled with python3 can't be opened in python2

Python 2.7.12 (default, Jul  1 2016, 15:12:24)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open('3.pkl', 'r') as picklefile:
...     print(pickle.load(picklefile))
... 
Traceback (most recent call last):
[...]
File "/usr/lib/python2.7/pickle.py", line 892, in load_proto
raise ValueError, "unsupported pickle protocol: %d" % proto
ValueError: unsupported pickle protocol: 3

So, upgrading entirely to python 3.5+ doesn't offer any difficulties related to pickling (old files can still be loaded), but maintaining a backwards-compatible codebase will require python 2.7 protocols only (new files cannot be loaded).

PIL

In changing from 2.7 to 3.5+, the PIL dependency had to be changed to a dependency on the pillow library, which is a python 3 compatible fork of PIL.

All other dependencies are the same.