Doc build failure - unicode error in generate_example_rst

[mdboom]

On Python 2, io.open() is called without an encoding argument. The default encoding it uses depends on the locale, so on Launchpad buildbots using the C locale, the default encoding is ASCII, and it chokes on any non-ascii character.

(This platform-dependent default is really annoying for anything cross platform, but it's part of the tension between Python as a serious application language and Python as a quick scripting language)

Exception occurred:
  File "/usr/lib/python2.7/dist-packages/sphinx/cmdline.py", line 188, in main
    warningiserror, tags)
  File "/usr/lib/python2.7/dist-packages/sphinx/application.py", line 134, in __init__
    self._init_builder(buildername)
  File "/usr/lib/python2.7/dist-packages/sphinx/application.py", line 195, in _init_builder
    self.emit('builder-inited')
  File "/usr/lib/python2.7/dist-packages/sphinx/application.py", line 314, in emit
    results.append(callback(self, *args))
  File "/build/buildd/matplotlib-1.2.0~1+6540+23~raring1/doc/sphinxext/gen_rst.py", line 44, in generate_example_rst
    contents = io.open(fullpath).read()
  File "/usr/lib/python2.7/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]

Buildlog: https://launchpadlibrarian.net/137176232/buildlog_ubuntu-raring-i386.matplotlib_1.2.0~1%2B6540%2B23~raring1_FAILEDTOBUILD.txt.gz

[pelson]

Thanks @takluyver, is the solution to fix the encoding, or is it more tricky than that? If it is a trivial fix, would you be willing to submit a PR?

It'd be really nice to get this fixed for v1.3, but I've not put this under the v1.3.x blocker milestone.

Cheers,

[mdboom]

All of the matplotlib source files are utf-8 encoded (when not ascii), so I think it's fine to just hardcode the encoding as utf-8 here. I'll attach a PR shortly, and then maybe @takluyver, you could confirm it works in the Launchpad environment?

I'm re-milestoning this as 1.2.x, since the bug exists there, too.

[takluyver]

Should be fine to hardcode the encoding (the code already does for Python 3, but the docs are built in 2.7).

I'm happy to test in Launchpad, but AFAIK that's only possible once it's been merged. In the meantime, I can replicate the failure locally by setting LANG=C before building the docs.

[mdboom]

As for milestoning -- I'm putting this back on 1.3.x blocker as @pelson had it originally. I've made it against the 1.2.x branch, but I think it's probably more important that we resolve this before putting out 1.3.x. (If only we could assign multiple milestones...)

[pelson]

👍 LGTG once @takluyver's tested.

[takluyver]

Yep, seems to do the trick. I haven't got time now to run the whole docs build, but it gets past where it was failing before.

[takluyver]

Ping - I think this can be merged, if you're happy with it.

[pelson]

I still get failures:

preparing documents... done
writing output... [  0%] api/artist_api                                                                                                               
Exception occurred:
  File "/usr/local/lib/python2.7/site-packages/Sphinx-1.1.3-py2.7.egg/sphinx/ext/graphviz.py", line 219, in render_dot_html
    mapfile = open(outfn + '.map', 'rb')
IOError: [Errno 2] No such file or directory: 'matplotlib/doc/build/html/_images/inheritance-b7a187684f018837b00a23f9b8b653b1c66f5f88.png.map'

But the problem being described is certainly fixed by this change, so I'm going to go ahead and merge.

@mdboom - I'm guessing you're able to build the docs without a hitch?

[takluyver]

Launchpad builds are working again, and I've switched it back to daily builds: https://code.launchpad.net/~takluyver/+recipe/matplotlib-daily

[pelson]

Lovely. Thanks @takluyver. Is there a link to the documentation which is part of the build, I can't find it for some reason.

[takluyver]

It doesn't actually publish the resulting documentation on the web, but it makes a Debian package from it, which you can install on Debian-based systems or unpack with an archive tool on other systems. The package itself doesn't have a stable URL (the latest version is here), but it's always accessible from this page.

Mostly I just look out for e-mails saying that it failed, though. I can't see a way for other people to subscribe to those e-mails, but I've posted a question to see if that's possible.