Make make.py build docs under python 3

[dmcdougall]

Fixes #1356.

@cjmayo Could you confirm this works as intended?

[pelson]

+1 Good stuff (not tested, but looks fine to me, and is backwards compatible to python2.6)

[mdboom]

It also needs

raise Error, errors

updated to

raise Error(errors)

on line 105.

Other than that, this looks good.

[cjmayo]

Looks like "as" was only introduced in Python 2.6:
http://docs.python.org/release/2.6.7/whatsnew/2.6.html#pep-3110-exception-handling-changes

because this will break pre 2.6 is it simpler to follow the note in make.py and replace the entire compatibility block with:
from shutil import ignore_patterns, copytree

which gets make.py working for me on Python 3.2. But there were a couple of other files in the document build system I had to patch too that I include below. And also 2to3 misses removing the exceptions module from the import in lib/matplotlib/sphinxext/plot_directive.py (it's after a line continuation character).

I can't verify the documents output though as my document build breaks with:
python3.2: CXX/Python3/cxx_extensions.cxx:1138: virtual Py::PythonExtensionBase::~PythonExtensionBase(): Assertion `ob_refcnt == 0' failed.

Don't think that is related and I'm still investigating, I don't have all the Python modules installed to support all the various back-ends.

--- doc/conf.py.orig
+++ doc/conf.py
@@ -174,10 +174,10 @@

 # Additional stuff for the LaTeX preamble.
 latex_preamble = """
-   \usepackage{amsmath}
-   \usepackage{amsfonts}
-   \usepackage{amssymb}
-   \usepackage{txfonts}
+   \\usepackage{amsmath}
+   \\usepackage{amsfonts}
+   \\usepackage{amssymb}
+   \\usepackage{txfonts}
 """

 # Documents to append as an appendix to all manuals.

--- doc/sphinxext/gen_rst.py.orig
+++ doc/sphinxext/gen_rst.py
@@ -37,15 +37,15 @@
                 continue

             fullpath = os.path.join(root,fname)
-            contents = file(fullpath).read()
+            contents = open(fullpath, encoding='utf8').read()
             # indent
             relpath = os.path.split(root)[-1]
             datad.setdefault(relpath, []).append((fullpath, fname, contents))

-    subdirs = datad.keys()
+    subdirs = list(datad.keys())
     subdirs.sort()

-    fhindex = file(os.path.join(exampledir, 'index.rst'), 'w')
+    fhindex = open(os.path.join(exampledir, 'index.rst'), 'w')
     fhindex.write("""\
 .. _examples-index:

@@ -77,7 +77,7 @@
             os.makedirs(outputdir)

         subdirIndexFile = os.path.join(rstdir, 'index.rst')
-        fhsubdirIndex = file(subdirIndexFile, 'w')
+        fhsubdirIndex = open(subdirIndexFile, 'w')
         fhindex.write('    %s/index.rst\n\n'%subdir)

         fhsubdirIndex.write("""\
@@ -122,14 +122,14 @@
                                   ) and
                        not noplot_regex.search(contents))
             if not do_plot:
-                fhstatic = file(outputfile, 'w')
+                fhstatic = open(outputfile, 'w')
                 fhstatic.write(contents)
                 fhstatic.close()

             if not out_of_date(fullpath, outrstfile):
                 continue

-            fh = file(outrstfile, 'w')
+            fh = open(outrstfile, 'w', encoding='utf8')
             fh.write('.. _%s-%s:\n\n'%(subdir, basename))
             title = '%s example code: %s'%(subdir, fname)
             #title = '<img src=%s> %s example code: %s'%(thumbfile, subdir, fname)

[WeatherGod]

Note: We don't support pre-2.6 anymore.

[dmcdougall]

It also needs
raise Error, errors
updated to
raise Error(errors)
on line 105.

On it.

Looks like "as" was only introduced in Python 2.6:
http://docs.python.org/release/2.6.7/whatsnew/2.6.html#pep-3110-exception-handling-changes

because this will break pre 2.6 is it simpler to follow the note in make.py and replace the entire compatibility block >with:
from shutil import ignore_patterns, copytree

As Ben said, python 2.5 and before are not supported.

Cheers for the patches, @cjmayo. It looks like this is ready to merge once I push @mdboom's suggestion. I think I will put the other doc fixes into another pull request so they don't hold this one up. Thanks again.

[dmcdougall]

@cjmayo Those patches break doc building with python 2.7 (and possibly 2.6, too) the encoding kwarg is not a valid kwarg for the function open. I'll need to think about how best to do this.

[cjmayo]

On the import exceptions in lib/matplotlib/sphinxext/plot_directive.py. It looks like 2to3 just does not remove exceptions from the import list. There is an upstream bug that says its fixed:
http://bugs.python.org/issue2350

but I have raised a new one as it doesn't work for me in 2.7,3.2 or 3.3:
http://bugs.python.org/issue16214

[dmcdougall]

@cjmayo Weird. The reproducible example you gave in that bug report (I haven't tried it) couldn't be simpler.

[dmcdougall]

Looks like the codec module will work for py2/py3 file encoding compatibilities.

[dmcdougall]

Ended up using io. See #1383.

[dmcdougall]

These commits have been moved to #1383 to keep everything in one place.