Skip to content
This repository

Crash python with matplotlib and unequal length arrays #833

Closed
richardeverson opened this Issue October 01, 2011 · 11 comments

3 participants

richardeverson Min RK Fernando Perez
richardeverson

The following code causes ipython to crash when imported with "import breaker"

breaker.py
--------------------------------------------------------------------

import numpy
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt


fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')

Y = numpy.random.rand(20, 3)

I = [2, 3, 5]
# Error here: different length arrays for the arguements of scatter
ax.scatter(Y[I,0], Y[I,1], Y[:,2], c='r', marker='o')

---------------------------------------------------------------------------------------------

Here's the crash report


***************************************************************************

IPython post-mortem report

{'commit_hash': '4eb31c8',
 'commit_source': 'repository',
 'ipython_path': '/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython',
 'ipython_version': '0.11',
 'os_name': 'posix',
 'platform': 'Darwin-11.1.0-x86_64-i386-64bit',
 'sys_executable': '/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python',
 'sys_platform': 'darwin',
 'sys_version': '2.7.2 (default, Sep  3 2011, 01:14:13) \n[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]'}

***************************************************************************



***************************************************************************

Crash traceback:

---------------------------------------------------------------------------
TypeErrorPython 2.7.2: /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
                                                   Sat Oct  1 23:06:48 2011
A problem occured executing Python code.  Here is the sequence of function
calls leading up to the error, with the most recent (innermost) call last.
/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/artist.py in draw_wrapper(artist=<matplotlib.figure.Figure object>, renderer=<matplotlib.backends.backend_macosx.RendererMac instance>, *args=(), **kwargs={})
     40         if artist.get_agg_filter() is not None:
     41             renderer.start_filter()
     42 
     43 
     44     def after(artist, renderer):
     45 
     46         if artist.get_agg_filter() is not None:
     47             renderer.stop_filter(artist.get_agg_filter())
     48 
     49         if artist.get_rasterized():
     50             renderer.stop_rasterizing()
     51 
     52     # the axes class has a second argument inframe for its draw method.
     53     def draw_wrapper(artist, renderer, *args, **kwargs):
     54         before(artist, renderer)
---> 55         draw(artist, renderer, *args, **kwargs)
        global draw = undefined
        artist = <matplotlib.figure.Figure object at 0x1012ca510>
        renderer = <matplotlib.backends.backend_macosx.RendererMac instance at 0x1038f14d0>
        args = ()
        kwargs = {}
     56         after(artist, renderer)
     57 
     58     # "safe wrapping" to exactly replicate anything we haven't overridden above
     59     draw_wrapper.__name__ = draw.__name__
     60     draw_wrapper.__dict__ = draw.__dict__
     61     draw_wrapper.__doc__  = draw.__doc__
     62     draw_wrapper._supports_rasterization = True
     63     return draw_wrapper
     64 
     65 
     66 class Artist(object):
     67     """
     68     Abstract base class for someone who renders into a
     69     :class:`FigureCanvas`.
     70     """

/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/figure.py in draw(self=<matplotlib.figure.Figure object>, renderer=<matplotlib.backends.backend_macosx.RendererMac instance>)
    869             dsu.append((self.images[0].get_zorder(), draw_composite, []))
    870 
    871         # render the axes
    872         for a in self.axes:
    873             dsu.append( (a.get_zorder(), a.draw, [renderer]))
    874 
    875         # render the figure text
    876         for a in self.texts:
    877             dsu.append( (a.get_zorder(), a.draw, [renderer]))
    878 
    879         for a in self.legends:
    880             dsu.append( (a.get_zorder(), a.draw, [renderer]))
    881 
    882         dsu.sort(key=itemgetter(0))
    883         for zorder, func, args in dsu:
--> 884             func(*args)
        func = <bound method Axes3DSubplot.draw of <matplotlib.axes.Axes3DSubplot object at 0x1038f33d0>>
        args = [<matplotlib.backends.backend_macosx.RendererMac instance at 0x1038f14d0>]
    885 
    886         renderer.close_group('figure')
    887 
    888         self._cachedRenderer = renderer
    889 
    890         self.canvas.draw_event(renderer)
    891 
    892     def draw_artist(self, a):
    893         """
    894         draw :class:`matplotlib.artist.Artist` instance *a* only --
    895         this is available only after the figure is drawn
    896         """
    897         assert self._cachedRenderer is not None
    898         a.draw(self._cachedRenderer)
    899 

/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py in draw(self=<matplotlib.axes.Axes3DSubplot object>, renderer=<matplotlib.backends.backend_macosx.RendererMac instance>)
    174 
    175     def draw(self, renderer):
    176         # draw the background patch
    177         self.axesPatch.draw(renderer)
    178         self._frameon = False
    179 
    180         # add the projection matrix to the renderer
    181         self.M = self.get_proj()
    182         renderer.M = self.M
    183         renderer.vvec = self.vvec
    184         renderer.eye = self.eye
    185         renderer.get_axis_position = self.get_axis_position
    186 
    187         # Calculate projection of collections and zorder them
    188         zlist = [(col.do_3d_projection(renderer), col) \
--> 189                  for col in self.collections]
        col = <mpl_toolkits.mplot3d.art3d.Patch3DCollection object at 0x1031842d0>
        self.collections = [<mpl_toolkits.mplot3d.art3d.Patch3DCollection object at 0x1031842d0>]
    190         zlist.sort()
    191         zlist.reverse()
    192         for i, (z, col) in enumerate(zlist):
    193             col.zorder = i
    194 
    195         # Calculate projection of patches and zorder them
    196         zlist = [(patch.do_3d_projection(renderer), patch) \
    197                 for patch in self.patches]
    198         zlist.sort()
    199         zlist.reverse()
    200         for i, (z, patch) in enumerate(zlist):
    201             patch.zorder = i
    202 
    203         if self._axis3don:
    204             axes = (self.xaxis, self.yaxis, self.zaxis)

/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/art3d.py in do_3d_projection(self=<mpl_toolkits.mplot3d.art3d.Patch3DCollection object>, renderer=<matplotlib.backends.backend_macosx.RendererMac instance>)
    300         self._sort_zpos = val
    301 
    302     def set_3d_properties(self, zs, zdir):
    303         offsets = self.get_offsets()
    304         if len(offsets) > 0:
    305             xs, ys = zip(*self.get_offsets())
    306         else:
    307             xs = [0] * len(zs)
    308             ys = [0] * len(zs)
    309         self._offsets3d = juggle_axes(xs, ys, zs, zdir)
    310         self._facecolor3d = self.get_facecolor()
    311         self._edgecolor3d = self.get_edgecolor()
    312 
    313     def do_3d_projection(self, renderer):
    314         xs, ys, zs = self._offsets3d
--> 315         vxs, vys, vzs, vis = proj3d.proj_transform_clip(xs, ys, zs, renderer.M)
        vxs = undefined
        vys = undefined
        vzs = undefined
        vis = undefined
        global proj3d.proj_transform_clip = <function proj_transform_clip at 0x10390bc08>
        xs = (0.36216574028503989, 0.88035402678580754, 0.65598466579038128)
        ys = (0.71548712817891746, 0.37297349593277895, 0.42046023574406355)
        zs = array([ 0.29958288,  0.58103946,  0.48966569,  0.94888056,  0.81767847,
        0.88920983,  0.04094774,  0.39601193,  0.80754427,  0.21510721,
        0.25184705,  0.94699405,  0.3357232 ,  0.98106279,  0.30822945,
        0.53472046,  0.81359326,  0.27006985,  0.57919094,  0.25529147])
        renderer.M = array([[  1.67125623,   1.45979591,   0.        ,  -1.83274964],
       [ -0.48245012,   1.26422034,   0.92119087,  -0.85903352],
       [  0.        ,   0.        ,   0.        , -10.        ],
       [ -0.83562812,   2.18969387,  -0.5318498 ,   9.5992225 ]])
    316         #FIXME: mpl allows us no way to unset the collection alpha value
    317         self._alpha = None
    318         self.set_facecolors(zalpha(self._facecolor3d, vzs))
    319         self.set_edgecolors(zalpha(self._edgecolor3d, vzs))
    320         PatchCollection.set_offsets(self, zip(vxs, vys))
    321 
    322         if vzs.size > 0 :
    323             return min(vzs)
    324         else :
    325             return np.nan
    326 
    327     def draw(self, renderer):
    328         self._old_draw(renderer)
    329 
    330 def patch_collection_2d_to_3d(col, zs=0, zdir='z'):

/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/proj3d.py in proj_transform_clip(xs=(0.36216574028503989, 0.88035402678580754, 0.65598466579038128), ys=(0.71548712817891746, 0.37297349593277895, 0.42046023574406355), zs=array([ 0.29958288,  0.58103946,  0.48966569,  0...81359326,  0.27006985,  0.57919094,  0.25529147]), M=array([[  1.67125623,   1.45979591,   0.        ...2812,   2.18969387,  -0.5318498 ,   9.5992225 ]]))
    188 
    189 def proj_transform(xs, ys, zs, M):
    190     """
    191     Transform the points by the projection matrix
    192     """
    193     vec = vec_pad_ones(xs, ys, zs)
    194     return proj_transform_vec(vec, M)
    195 
    196 def proj_transform_clip(xs, ys, zs, M):
    197     """
    198     Transform the points by the projection matrix
    199     and return the clipping result
    200     returns txs,tys,tzs,tis
    201     """
    202     vec = vec_pad_ones(xs, ys, zs)
--> 203     return proj_transform_vec_clip(vec, M)
        global proj_transform_vec_clip = <function proj_transform_vec_clip at 0x10390ba28>
        vec = array([(0.36216574028503989, 0.88035402678580754, 0.65598466579038128),
       (0.71548712817891746, 0.37297349593277895, 0.42046023574406355),
       [ 0.29958288  0.58103946  0.48966569  0.94888056  0.81767847  0.88920983
  0.04094774  0.39601193  0.80754427  0.21510721  0.25184705  0.94699405
  0.3357232   0.98106279  0.30822945  0.53472046  0.81359326  0.27006985
  0.57919094  0.25529147],
       [ 1.  1.  1.]], dtype=object)
        M = array([[  1.67125623,   1.45979591,   0.        ,  -1.83274964],
       [ -0.48245012,   1.26422034,   0.92119087,  -0.85903352],
       [  0.        ,   0.        ,   0.        , -10.        ],
       [ -0.83562812,   2.18969387,  -0.5318498 ,   9.5992225 ]])
    204 transform = proj_transform
    205 
    206 def proj_points(points, M):
    207     return zip(*proj_trans_points(points, M))
    208 
    209 def proj_trans_points(points, M):
    210     xs, ys, zs = zip(*points)
    211     return proj_transform(xs, ys, zs, M)
    212 
    213 def proj_trans_clip_points(points, M):
    214     xs, ys, zs = zip(*points)
    215     return proj_transform_clip(xs, ys, zs, M)
    216 
    217 def test_proj_draw_axes(M, s=1):
    218     import pylab

/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/proj3d.py in proj_transform_vec_clip(vec=array([(0.36216574028503989, 0.88035402678580754...0.25529147],
       [ 1.  1.  1.]], dtype=object), M=array([[  1.67125623,   1.45979591,   0.        ...2812,   2.18969387,  -0.5318498 ,   9.5992225 ]]))
    145     b = -2*(zfront*zback)/(zfront-zback)
    146     return np.array([[1,0,0,0],
    147                      [0,1,0,0],
    148                      [0,0,a,b],
    149                      [0,0,-1,0]
    150                      ])
    151 
    152 def proj_transform_vec(vec, M):
    153     vecw = np.dot(M, vec)
    154     w = vecw[3]
    155     # clip here..
    156     txs, tys, tzs = vecw[0]/w, vecw[1]/w, vecw[2]/w
    157     return txs, tys, tzs
    158 
    159 def proj_transform_vec_clip(vec, M):
--> 160     vecw = np.dot(M, vec)
        vecw = undefined
        global np.dot = <built-in function dot>
        M = array([[  1.67125623,   1.45979591,   0.        ,  -1.83274964],
       [ -0.48245012,   1.26422034,   0.92119087,  -0.85903352],
       [  0.        ,   0.        ,   0.        , -10.        ],
       [ -0.83562812,   2.18969387,  -0.5318498 ,   9.5992225 ]])
        vec = array([(0.36216574028503989, 0.88035402678580754, 0.65598466579038128),
       (0.71548712817891746, 0.37297349593277895, 0.42046023574406355),
       [ 0.29958288  0.58103946  0.48966569  0.94888056  0.81767847  0.88920983
  0.04094774  0.39601193  0.80754427  0.21510721  0.25184705  0.94699405
  0.3357232   0.98106279  0.30822945  0.53472046  0.81359326  0.27006985
  0.57919094  0.25529147],
       [ 1.  1.  1.]], dtype=object)
    161     w = vecw[3]
    162     # clip here..
    163     txs, tys, tzs = vecw[0]/w, vecw[1]/w, vecw[2]/w
    164     tis = (vecw[0] >= 0) * (vecw[0] <= 1) * (vecw[1] >= 0) * (vecw[1] <= 1)
    165     if np.sometrue(tis):
    166         tis =  vecw[1] < 1
    167     return txs, tys, tzs, tis
    168 
    169 def inv_transform(xs, ys, zs, M):
    170     iM = linalg.inv(M)
    171     vec = vec_pad_ones(xs, ys, zs)
    172     vecr = np.dot(iM, vec)
    173     try:
    174         vecr = vecr/vecr[3]
    175     except OverflowError:

TypeError: can't multiply sequence by non-int of type 'float'

***************************************************************************

History of session input:import breakerimport breakerimport breakerimport breakerhelp(numpy.random.rand)import breaker
*** Last line of input (may not be in above history):
import breaker

Min RK
Owner

Another instance of our crash handler being overzealous. Should probably be a modest fix.

Min RK
Owner

What matplotlib version and backend are you using? With current master (1.2.x), I get a regular traceback on this one, so this may have been fixed in the 1.1 release.

Fernando Perez
Owner

@minrk, I was able to reproduce the giant traceback with mpl 1.1.1.

Min RK
Owner

@fperez - what backend?

richardeverson

I'm using mpl 1.1.0. With the MacOS backend I get the giant traceback, but with TkAgg a normal traceback:

In [3]: matplotlib.version
Out[3]: '1.1.0'

In [4]: rcParams['backend']
Out[4]: 'TkAgg'

In [5]: import breaker

In [6]: Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 1410, in call
return self.func(*args)
File "/usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 495, in callit
func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/backends/backend_tkagg.py", line 254, in idle_draw
self.draw()
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/backends/backend_tkagg.py", line 239, in draw
FigureCanvasAgg.draw(self)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/backends/backend_agg.py", line 401, in draw
self.figure.draw(self.renderer)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/artist.py", line 55, in draw_wrapper
draw(artist, renderer, *args, **kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/matplotlib/figure.py", line 884, in draw
func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/axes3d.py", line 189, in draw
for col in self.collections]
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/art3d.py", line 315, in do_3d_projection
vxs, vys, vzs, vis = proj3d.proj_transform_clip(xs, ys, zs, renderer.M)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/proj3d.py", line 203, in proj_transform_clip
return proj_transform_vec_clip(vec, M)
File "/System/Library/Frameworks/Python.framework/Versions/src/matplotlib/lib/mpl_toolkits/mplot3d/proj3d.py", line 160, in proj_transform_vec_clip
vecw = np.dot(M, vec)
TypeError: can't multiply sequence by non-int of type 'float'

In [6]:

I'll try the more recent versions.

Min RK
Owner

I do not see the crash with current master matplotlib and IPython on:

  • Ubuntu (10.04, default Python 2.6.5): tk, qt (PyQt4)
  • Mac OSX (10.7.2, System Python 2.7.1): osx, tk, qt (PySide)

In these five cases (10, counting the fact that the same behavior is seen in the qtconsole), I see a regular traceback.

The only situation where I can reproduce the crash traceback is with the gtk backend (both terminal and qtconsole).

Min RK
Owner

In any case, we have a general issue: The IPython crash handler should only be catching errors in IPython code. That means that when we hook up to an eventloop, we have to disable the crash handler immediately, and add enabling/disabling the crash handler as part of the callback we inject to the loop, so it is only active during IPython execution.

Fernando Perez
Owner

I get it with the Qt4agg backend on ubuntu 10.10, mpl current git.

But on the more general issue, I completely agree with you, the crash handler needs to get out of the way when event loops are there.

I'm actually inclined to go forward, and consider disabling the crash handler by default, and instead add a magic that turns it manually on. If people come to us with normal tracebacks that we can't debug and their crash is reproducible, we can tell them to type

%crash_handler on

and then to mail us the output.

I wrote that crash handler code early in ipython's life, when there was zero test suite and basically every problem I fixed was via crashes reported by users, so having this information was extremely useful. But in recent years, with increasingly solid test suite coverage (not perfect, but certainly decent), it seems like the CH is actually causing us more headaches than solutions. What do you think?

Min RK
Owner

I'm inclined to agree. Do we want to still override excepthook with something that mentions how to report issues or turn on the verbose handler, but without the big crash message, and massive traceback?

e.g.

-------------------------
regular traceback
...
Exception : msg

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to ipython-dev@scipy.org

You can enable a much more verbose traceback with `%crash_handler on`.
Fernando Perez
Owner

That sounds like the perfect solution to me, yes!

Fernando Perez fperez closed this November 20, 2011
Fernando Perez
Owner

Actually I'm closing this: it's just a bad call to matplotlib that only triggers in the event loop. The user did make an error, there's nothing we can do other than printing the traceback. The fact that the traceback looks different is an indicator that it came from somewhere outside of the normal execution. Users can still use %tb to get a detailed traceback and %debug also works, so I don't see what else we need to do here.

Fernando Perez fperez referenced this issue from a commit January 10, 2012
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.