_png.read_png crashes on Python 3 with urllib.request object

[mdboom]

The following code crashes on Python 3.x but not on Python 2.x (verified on Windows with mpl 1.2.0):

url = 'http://www.libpng.org/pub/png/img_png/pngnow.png'
try:
    import urllib2
    data = urllib2.urlopen(url)
except Exception:
    import urllib.request
    data = urllib.request.urlopen(url)

from matplotlib import pyplot

image = pyplot.imread(data)  # crash on py3.x
pyplot.imshow(image)
pyplot.show()

The crash is at _png.cpp line 328

Maybe using numpy's npy_PyFile_* functions could help. A lot of thought has gone into those functions.

[mdboom]

Thanks for the tip about npy_PyFile. I wasn't aware of those. Using those does seem to fix things (and simplifies our code). Once you've confirmed this works for you, I'll go ahead and merge. The merge back to master is going to be tricky, as the _png extension has been largely rewritten over there.

[cgohlke]

Thanks, that was fast. It does work for me. Verified with matplotlib-1.2.0.win-amd64-py3.3.exe.
Does this change the numpy version required by matplotlib?

[mdboom]

Ugh... I didn't realise -- it appears that npy_PyFile_CloseFile is only in Numpy 1.7. That's not going to fly. The other functions go back to 1.5.0, which is better, but still higher than the 1.4.0 which is our current minimum requirement.

I guess we copy these over to our own tree? I'd hate to do that...

[cgohlke]

This needs more testing. In IPython 0.13.1 notebook on Python 3.3 now I get:

In [5]: pyplot.plot()
Out[5]:
[]
---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
X:\Python33\lib\site-packages\IPython\zmq\pylab\backend_inline.py in show(close)
    100     try:
    101         for figure_manager in Gcf.get_all_fig_managers():
--> 102             send_figure(figure_manager.canvas.figure)
    103     finally:
    104         show._to_draw = []

X:\Python33\lib\site-packages\IPython\zmq\pylab\backend_inline.py in send_figure(fig)
    209     """
    210     fmt = InlineBackend.instance().figure_format
--> 211     data = print_figure(fig, fmt)
    212     # print_figure will return None if there's nothing to draw:
    213     if data is None:

X:\Python33\lib\site-packages\IPython\core\pylabtools.py in print_figure(fig, fmt)
    102     try:
    103         bytes_io = BytesIO()
--> 104         fig.canvas.print_figure(bytes_io, format=fmt, bbox_inches='tight')
    105         data = bytes_io.getvalue()
    106     finally:

X:\Python33\lib\site-packages\matplotlib\backend_bases.py in print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, **kwargs)
   2050                     orientation=orientation,
   2051                     dryrun=True,
-> 2052                     **kwargs)
   2053                 renderer = self.figure._cachedRenderer
   2054                 bbox_inches = self.figure.get_tightbbox(renderer)

X:\Python33\lib\site-packages\matplotlib\backends\backend_agg.py in print_png(self, filename_or_obj, *args, **kwargs)
    501             _png.write_png(renderer._renderer.buffer_rgba(),
    502                            renderer.width, renderer.height,
--> 503                            filename_or_obj, self.figure.dpi)
    504         finally:
    505             if close:

UnsupportedOperation: fileno

[mdboom]

Ok -- seems to be reproduceable on Python 3.2 as well, with IPython 0.13. IPython git master fails to display inline plots on Python 3.x for an apparently unrelated reason.

In any case, I have a fix for this. When a "fake" file object (such as a BytesIO here) is passed in, PyFile_Dup will fail and set an exception. This exception needs to be cleared because we have a fallback method for handling these fake files.

[cgohlke]

I just noticed that files are not opened in binary mode for reading and writing. I think for Windows the file open modes must be "rb" and "wb" (maybe I am missing something).

[mdboom]

Good catch. I've updated the PR to open the files in binary mode.

[pelson]

@mdboom - have you done a merge back to master recently? I haven't - it's probably a good idea to get on top of that hurdle as soon as we can if you haven't either.

[mdboom]

I did this a few times yesterday, and it seems we're caught up at the moment. I agree with you -- it's usually best to merge as soon as possible to avoid large, complex conflicts.