memoryviews of arrays of structs fail to work in some cases #1407
Description
As reported on the mailing list:
There seem to be some bugs regarding memoryviews of structured arrays
in Cython when the struct itself contains arrays. I have a Cython fix for
one of them (the most relevant to me), but first I would like to make sure
that these are really bugs and I am not doing anything stupid. This
question is related to
http://stackoverflow.com/questions/17239091/cython-memoryviews-from-array-of-structs,
but the answer there works only for char arrays. An array of doubles breaks
it again.
It seems to me that Cython implementation solves some special cases, but
not the general one. My proposed fix would be to insert
ndim = ctx->head->field->type->ndim;
after
/* Process the previous element */
if (__Pyx_BufFmt_ProcessTypeChunk(ctx) == -1) return NULL;
in Cython\Utility\Buffer.c
This fixes tests 2, 4 and 7, but not 8 and 9 (see below). I am not sure
about the details of what the code in Buffer.c is supposed to do and
whether my modification would break some other functionality, but my code
(not only the tests) works with this fix.
Or am I possibly simply missing "the right way" to handle arrays of structs
containing arrays in Cython? But in either case the compiler shouldn't
crash like in test 9...
Here is the code of my tests with and without numpy with results ranging
from "ok" to "compiler crash".
The numpy tests require these imports:
import numpy as np
cimport numpy as np
*Test 1 (OK)*
cdef packed struct My_Struct:
double my_first_value
double my_second_value
cpdef test():
cdef My_Struct my_struct
cdef My_Struct[my_view = <My_Struct[:1](:])>&my_struct
*Test 2 (FAILS)*
cdef packed struct My_Struct:
double my_first_value
double my_second_value*[test():
cdef My_Struct my_struct
cdef My_Struct[:](1]*
cpdef) my_view = <My_Struct[Expected 1 dimension(s), got 1*
*
*
*Test 3 (OK)*
cdef packed struct My_Struct:
double my_first_value
double my_second_value[*1*](:1]>&my_struct
*ValueError:)
cpdef test():
cdef np.ndarray my_array = np.ndarray(1,
dtype=np.dtype([('my_second_value','float64',*
1*)](('my_first_value','float64'),), align=False)
cdef My_Struct[my_view = my_array
*Test 4 (FAILS)*
cdef packed struct My_Struct:
double my_first_value
double my_second_value[*2*](:])
cpdef test():
cdef np.ndarray my_array = np.ndarray(1,
dtype=np.dtype([('my_second_value','float64',*
2*)](('my_first_value','float64'),), align=False)
cdef My_Struct[my_view = my_array
*ValueError: Expected 1 dimension(s), got 1*
*
*
*Test 5 (OK)*
cdef packed struct My_Struct:
double my_first_value
*char* my_second_value[2](:])
cpdef test():
cdef np.ndarray my_array = np.ndarray(1,
dtype=np.dtype([('my_second_value','*a*',2)](('my_first_value','float64'),),
align=False)
cdef My_Struct[my_view = my_array
*Test 6 (OK)*
cdef packed struct My_Array_Struct:
double values[*1*](:])
cdef packed struct My_Struct:
double my_first_value
My_Array_Struct my_second_value
cpdef test():
cdef np.ndarray my_array = np.ndarray(1,
dtype=np.dtype([('my_second_value','float64',*
1*)](('my_first_value','float64'),), align=False)
cdef My_Struct[my_view = my_array
*Test 7 (FAILS)*
cdef packed struct My_Array_Struct:
double values[*2*](:])
cdef packed struct My_Struct:
double my_first_value
My_Array_Struct my_second_value
cpdef test():
cdef np.ndarray my_array = np.ndarray(1,
dtype=np.dtype([('my_second_value','float64',*
2*)](('my_first_value','float64'),), align=False)
cdef My_Struct[my_view = my_array
*ValueError: Expected 1 dimension(s), got 1*
*Test 8 (FAILS)*
cdef packed struct My_Struct:
double my_first_value*[1](:])*
double my_second_value
cpdef test():
cdef My_Struct my_struct
cdef My_Struct[my_view = <My_Struct[:1](:])>&my_struct
*ValueError: Buffer dtype mismatch, expected 'double' but got end in
'My_Struct.my_second_value'*
*Test 9 (COMPILER CRASH)*
cdef packed struct My_Array_Struct:
double values[packed struct My_Struct:
double my_first_value
My_Array_Struct my_second_value*[1](2]
cdef)*
cpdef test():
cdef My_Struct my_struct
cdef My_Struct[my_view = <My_Struct[:1](:])>&my_struct
Migrated from http://trac.cython.org/ticket/853