BUG: AttributeError: incompatible shape for a non-contiguous array #2700

Closed
pmarshwx opened this Issue Oct 29, 2012 · 10 comments

Projects

None yet

5 participants

@pmarshwx

The following is an error when running the matplotlib test suite. The bug is traced back to this commit c48156d


Traceback (most recent call last):
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/nose-1.2.1-py2.7.egg/nose/case.py", line 197, in runTest
self.test(_self.arg)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/testing/decorators.py", line 39, in failer
result = f(_args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/testing/decorators.py", line 145, in do_test
figure.savefig(actual_fname)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/figure.py", line 1363, in savefig
self.canvas.print_figure(_args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 2118, in print_figure
*_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/backend_bases.py", line 1894, in print_svg
return svg.print_svg(_args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/backends/backend_svg.py", line 1103, in print_svg
return self._print_svg(filename, svgwriter, fh_to_close, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/backends/backend_svg.py", line 1139, in _print_svg
self.figure.draw(renderer)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/artist.py", line 54, in draw_wrapper
draw(artist, renderer, *args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/figure.py", line 999, in draw
func(_args)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/artist.py", line 54, in draw_wrapper
draw(artist, renderer, *args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/axes.py", line 2086, in draw
a.draw(renderer)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/artist.py", line 54, in draw_wrapper
draw(artist, renderer, _args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/collections.py", line 699, in draw
return Collection.draw(self, renderer)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/artist.py", line 54, in draw_wrapper
draw(artist, renderer, _args, *_kwargs)
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/collections.py", line 248, in draw
transform, transOffset, offsets, paths = self._prepare_points()
File "/home/Patrick.Marsh/.local/lib/python2.7/site-packages/matplotlib/collections.py", line 222, in _prepare_points
offsets.shape = (-1, 2) # Make it Nx2
AttributeError: incompatible shape for a non-contiguous array

@WeatherGod
Contributor

Further testing shows that this failure occurs when trying to reshape a (2,) shaped array to (1, 2). What is strange is that when done by hand, I don't get this error. Trying different permutations.

@seberg
Member
seberg commented Oct 29, 2012

Thanks for reporting, I have figured it out... This occurs for 0-sized arrays, the array you put in was probably not a (2,) array but a (0,2) array, and that commit changed that 0-sized arrays are both F- and C-contiguous always (I thought this was a bug that it was not the case before?). Took a while to figure out, but this should fix it:

--- a/numpy/core/src/multiarray/shape.c
+++ b/numpy/core/src/multiarray/shape.c
@@ -231,10 +231,10 @@ PyArray_Newshape(PyArrayObject *self, PyArray_Dims *newdim
          * data in the order it is in.
          */
         if (!(PyArray_ISONESEGMENT(self)) ||
-            (((PyArray_CHKFLAGS(self, NPY_ARRAY_C_CONTIGUOUS) &&
-               order == NPY_FORTRANORDER) ||
-              (PyArray_CHKFLAGS(self, NPY_ARRAY_F_CONTIGUOUS) &&
-                  order == NPY_CORDER)) && (PyArray_NDIM(self) > 1))) {
+            ((order == NPY_CORDER &&
+              !PyArray_CHKFLAGS(self, NPY_ARRAY_C_CONTIGUOUS)) ||
+             (order == NPY_FORTRANORDER &&
+              !PyArray_CHKFLAGS(self, NPY_ARRAY_F_CONTIGUOUS)))) {
             int success = 0;
             success = _attempt_nocopy_reshape(self, ndim, dimensions,
                                               newstrides, order);

I have to think about it if that cannot be simplified more... And probably have to look more closely for calls to PyArray_NDIM(...) > 1 to check if they are not quite right now. The thing is that _attempt_nocopy_reshape cannot handle 0-d arrays and due to the Flags fix for them, it was called when it should not have been called.

@WeatherGod
Contributor

On Monday, October 29, 2012, seberg wrote:

Thanks for reporting, I have figured it out... This occurs for 0-sized
arrays, the array you put in was probably not a (2,) array but a (0,2)
array, and that commit changed that 0-sized arrays are both F- and
C-contiguous always (I thought this was a bug that it was not the case
before?). Took a while to figure out, but this should fix it:

--- a/numpy/core/src/multiarray/shape.c+++ b/numpy/core/src/multiarray/shape.c@@ -231,10 +231,10 @@ PyArray_Newshape(PyArrayObject *self, PyArray_Dims *newdim
* data in the order it is in.
*/
if (!(PyArray_ISONESEGMENT(self)) ||- (((PyArray_CHKFLAGS(self, NPY_ARRAY_C_CONTIGUOUS) &&- order == NPY_FORTRANORDER) ||- (PyArray_CHKFLAGS(self, NPY_ARRAY_F_CONTIGUOUS) &&- order == NPY_CORDER)) && (PyArray_NDIM(self) > 1))) {+ ((order == NPY_CORDER &&+ !PyArray_CHKFLAGS(self, NPY_ARRAY_C_CONTIGUOUS)) ||+ (order == NPY_FORTRANORDER &&+ !PyArray_CHKFLAGS(self, NPY_ARRAY_F_CONTIGUOUS)))) {
int success = 0;
success = _attempt_nocopy_reshape(self, ndim, dimensions,
newstrides, order);

I have to think about it if that cannot be simplified more... And probably
have to look more closely for calls to PyArray_NDIM(...) > 1 to check if
they are not quite right now. The thing is that _attempt_nocopy_reshapecannot handle 0-d arrays and due to the Flags fix for them, it was called
when it should not have been called.

Hmmm, maybe that's why I couldn't reproduce it. I will double-check.

@WeatherGod
Contributor

Ok, narrowed it down a bit more. I originally placed a print statement at
the wrong place, which mislead me. It isn't 0-D arrays specifically, it is
empty N-D arrays (i.e., shape == (0, 2)), which is not a bug for us and is
perfectly valid. An array with shape (0,) works just fine. Some test code:

a = np.zeros((0, 2))
a.shape = (-1, 2)
@seberg
Member
seberg commented Oct 29, 2012

@WeatherGod thanks for double checking, thats what I meant sorry for writing the wrong thing the second time. The diff I posted will fix it then, I think it was probably not 100% correct before either, but almost always anyways.

@njsmith since you checked out that PR, just to make sure. There is no reason why a 0-sized array should not be both C- and F-Contiguous, is there? I thought that was not intentional, but so that at least someone else is aware, also have to check if its not necessary to document it clearer.

@njsmith
Member
njsmith commented Oct 30, 2012

Seems like an empty array should always be C- and F-contiguous to me...

@seberg seberg added a commit to seberg/numpy that referenced this issue Nov 1, 2012
@seberg seberg BUG: Reshape of 0-sized arrays failed to work without copy
This also adds a check for order=Keeporder which is not supported.
"closes Issue #2700"
237e816
@seberg
Member
seberg commented Jan 3, 2013

This is fixed in master.

@seberg seberg closed this Jan 3, 2013
@certik
Contributor
certik commented Jan 4, 2013

@seberg, is this fixed in 1.7 as well?

@seberg
Member
seberg commented Jan 4, 2013

Ah sorry should have noted. This was never an issue for 1.7. the bug only (surfaced) with changes that are not in 1.7. itself. 1.7. might have odd corner cases here, but they where there for a very long time.

@certik
Contributor
certik commented Jan 4, 2013

Ok, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment