Porting to Python 3, part 4 ("final") #558

In Python 2, type(float) returns "<type 'float'>", but in Python 3 it returns "<class 'float'>". This confuses the doctest which must match output exactly. Use the ellipsis syntax to avoid this. See also commit b457bee: Make some doctests Python 3 compatible.

In Python 3, floats have a higher precision by default and it is not possible to override it. This is a problem for doctests, which rely on matching output exactly. As the purpose of these doctests is to show limitations in Python floats, we can't just convert them to the more precise SymPy Floats. Fix by skipping the four affected doctests.

Commit f75a101 (Fix the doctest runner in Python 2.5 with utf-8 files) introduced the _indent method from upstream. In Python 3, this converts a (unicode) string to bytes, which don't support regex, thus crashing when there's a doctest failure (that's the only time this function is called). Wrap the encode code with a version check and enter it only under Python 2.

2to3 crashes when ran on a doctest containing reduce(). This is an upstream problem (issue 12611). Replace the only use of this in our doctests with a for cycle. This fixes issue sympy#2605.

In Python 3, the _load_testfile method has a fourth argument, encoding. As we already use this, just pass it to the function when under Python 3. However, we then try to decode a str (because we specified an encoding) which is wrong, so set the encoding to None in Python 3. We cannot just use None as an encoding as Python 2 relies on the decode step. This error only appeared when running .txt doctests, as the usual doctests use a different codepath.

2to3 reports some errors of the type "TokenError: ('EOF in multi-line statement', ...)" and can't parse those docstrings. Fix these by correctly using multi-line strings and statements in doctests.

In Python 2, a SymPy Float could be used as an argument of range(). This is no longer possible in Python 3. There are two occurances of this in wigner.py. In the first, a function which expected an integer (per the doctest) got a Float and then called range() on it; in the second, range() was called directly on Floats. Fix by explicitly casting to int.

In Python 3, variables are no longer in the namespace after being used in a list comprehension. Fix one doctest to directly use a number instead of a variable.

Some .txt doctests were setting the sys.displayhook to pprint which produced subtle errors in Python 3 (printing an extra "None" in some cases). Instead, use the init_printing() function from interactive.printing. It is called by default with use_unicode=false, to match the expected output in the files. These errors only appeared in tutorial.txt and matrices.txt, but use init_printing() in four other files which were also setting the displayhook. Also skip a doctest in the tutorial demonstrating the old-style division which no longer exists in Python 3 and modify two doctests in matrices.txt not to expect a result of None (which wasn't strictly correct).

The hasattr() function is defined to call getattr() and see if an exception is raised (in which case the attribute doesn't exist). This has been changed in Python 3.2 to catch only AttributeErrors and not all kinds of exceptions. See also issue sympy#2608. A property in printer.py checked for a key and so could raise a KeyError, which now wouldn't be caught by hasattr(). Refactor to raise an AttributeError if the key doesn't exist. This bug was caught by a test in utilities/tests/test_pickling.

The hasattr() function is defined to call getattr() and see if an exception is raised (in which case the attribute doesn't exist). This has been changed in Python 3.2 to catch only AttributeErrors and not all kinds of exceptions. See also issue sympy#2608 and the previous commit. The property D ("Dirac conjugation") could raise a ShapeError, which now wouldn't be caught by hasattr(). Wrap the matrix multiplication with a try/except statement that catches ShapeError and returns AttributeError instead. Also change a relevant test to expect an AttributeError now.

Some code was using the ambiguous division operator on integers. This produces floats by default in Python 3 (1.0 as opposed to 1) which cannot be used as list indices. Fix by using floor division (//) where appropriate. A doctest was also failing, it expected an implicit cast to int (1.0 to 1 automatically). Also remove some redundant casts to int, factorial() returns an int always.

SymPy needs a 2to3 run to be usable in Python 3. Unfortunately, running 2to3 on mpmath (which is already compatible) produces garbage code. To avoid this, we first copy all files (except those from mpmath) to a "sympy-py3k" directory, run 2to3 on them and then copy over the files from mpmath. One can then use SymPy normally from the sympy-py3k directory under Python 3. As running 2to3 is a slow process, we try to note which files have been updated since the last run and only process those. Also add "sympy-py3k" to .gitignore. Finally, this is a first draft of the script. As such, there are several TODOs which note areas of possible further improvement.

Commit 76d0046 added a function, findout_terminal_width. This function relies on regexing the stdout, which is of type str in Python 2. In Python 3 it is a 'bytes' object and it's not possible to regex it, which then crashes the doctest runner. Fix by checking if we are under Python 3 and decoding from "utf-8" if that is the case.

A doctest was using the ambiguous division operator (/). Because the expected result was floor division, use the unambiguous floor division operator (//). This removes errors under Python 3 and fixes issue sympy#2610.

The __init__.py file for quantum uses some importing magic so that both "from quantum import *" and "from quantum import State" to work. This fails under Python 3 because some underlying rules seem to have changed. The likely cause is that the module names shadow function names (or vice-versa). Fix by directly import __all__ from the affected modules. A better fix might be to change the module names, but this is a backward-incompatible change and this workaround is equivalent functionally.

Improve the explanation of the difference between SymPy numbers and Python numbers and other small changes.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Porting to Python 3, part 4 ("final") #558

Porting to Python 3, part 4 ("final") #558

Commits on Aug 19, 2011