numpy.pxd: No endianness or packing in buffer format string

[robertwb]

numpy.pxd should include endianness and packing information in the buffer format string, so that Cython can reject arrays which are not native endian/native packing.

At 2009-04-19T20:47:41Z @dagss added attachment patch1

Migrated from http://trac.cython.org/ticket/285

[robertwb]

@dagss changed component from Code Generation to Packaging
commented

I have no idea which component this should go under...

[robertwb]

@dagss commented

From the ML

Robert Bradshaw wrote:
> On Apr 17, 2009, at 1:45 PM, Kurt Smith wrote:
> 
>> The current numpy_test.pyx file fails for PowerPC macs due to
>> endianness issues in the dtype.  This is a small fix to make it work
>> (and make all tests pass on my machine).  It also adds an explicit
>> big-endian test to the doctest.
> 
> Hmm... This looks like a bug in the code itself, not just the test.

Thanks for bringing this up Kurt.

Using typed buffers in Cython only support the native endianness, so what one should get is an exception on one of the platforms.

It seems that the problem is that numpy.pxd does not include endianness in the formatting string; in fact it looks like this wasn't considered at all.

http://trac.cython.org/cython_trac/ticket/285

When it comes to the testcase, one should have two tests:

a) With native endianness, which should succeed. I think the best way is to modify the result:

x = ...
print(repr(x).replace(">","<")) # then follows little endian result

b) With non-native endianness, which should result in an exception from buffer acquisition.


> 
>> diff -r fc73225aaea1 tests/run/numpy_test.pyx
>> --- a/tests/run/numpy_test.pyx   Fri Apr 17 09:11:16 2009 +0200
>> +++ b/tests/run/numpy_test.pyx   Fri Apr 17 15:43:11 2009 -0500
>> @@ -132,13 +132,20 @@
>>>>> test_recordarray()
>>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([-            ('a', np.dtype('i,i')),\
>> -            ('b', np.dtype('i,i'))\
>> +            ('a', np.dtype('<i,<i')),\
>> +            ('b', np.dtype('<i,<i'))\
>>          ](\
>>))))
>>      array([0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))](((0,),
>>            dtype=[[('f0', '<i4'), ('f1', '<i4')](('a',)), ('b', ['<i4'), ('f1', '<i4')](('f0',
>>))])
>>
>>>>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([+            ('a', np.dtype('>i,>i')),\
>> +            ('b', np.dtype('>i,>i'))\
>> +        ](\
>>))))
>> +    array([0), (0, 0)), ((1, 2), (1, 4)), ((1, 2), (1, 4))](((0,),
>> +          dtype=[[('f0', '>i4'), ('f1', '>i4')](('a',)), ('b', ['>i4'), ('f1', '>i4')](('f0',
>>))])
>> +
>> +    >>> test_nested_dtypes(np.zeros((3,), dtype=np.dtype([             ('a', np.dtype('i,f')),\
>>              ('b', np.dtype('i,i'))\
>>          ](\
>>))))
>> @@ -297,6 +304,10 @@
>>      arr[= arr[0](1].b.x).a.y + 1
>>      arr[= 4
>>      arr[2](1].b.y) = arr[1]
>> +    if not arr.dtype.isnative:
>> +        # required to get the endianness
>> +        # indicator (< or >) correct in output.
>> +        arr.byteswap(True)
>>      return arr
>>
>>  def test_bad_nested_dtypes():
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev@codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
> 
> _______________________________________________
> Cython-dev mailing list
> Cython-dev@codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


-- 
Dag Sverre

[robertwb]

@dagss changed summary from

numpy.pxd does not include endianness information

to

numpy.pxd: No endianness in buffer format string
commented

[robertwb]

@dagss changed description from

numpy.pxd should include endianness information in the buffer format string, so that Cython can reject arrays which are not native endian.

to

numpy.pxd should include endianness and packing information in the buffer format string, so that Cython can reject arrays which are not native endian/native packing.
summary from

numpy.pxd: No endianness in buffer format string

to

numpy.pxd: No endianness or packing in buffer format string
commented

[robertwb]

@dagss commented

patch1 adds this to numpy.pxd, however Cython's buffer support doesn't support these format strings yet so I'm witholding the patch until then.

[robertwb]

@dagss changed resolution to fixed
status from new to closed
commented

[robertwb]

@dagss changed resolution from fixed to empty
status from closed to reopened
commented

[robertwb]

@dagss changed owner from somebody to dagss
status from reopened to new
commented

[robertwb]

@dagss changed resolution to fixed
status from new to closed
commented