Skip to content

_dtype_from_pep3118 is overly strict on prefixes #9049

Open
@bmerry

Description

@bmerry

In numpy 1.12.1, _dtype_from_pep3118 requires that modifiers are specified in the order shape then byte order then count, with at most one of each. The PEP is a bit vague, but it seems like modifiers ought to be interpreted recursively i.e. a modifier can be followed by any valid PEP 3118 specifier.

In practice this can lead to certain types failing to round-trip through a memoryview e.g.:

>>> dt = np.dtype({'formats': [np.dtype((np.dtype((np.int32, (3,))), (2,)))], 'names': ['foo']})
>>> a = np.empty(0, dt)
>>> m = memoryview(a)
>>> np.array(m)
NotImplementedError: memoryview: unsupported format T{(2)(3)i:foo:}

It also causes problems if one puts an endianness specifier at the start of the string (which should always be permitted, since the original struct module allows it). I haven't managed to get numpy to generate such a format string when wrapping an array into a memoryview, but it's causing me some problems while developing new features for pybind11:

>>> np.core._internal._dtype_from_pep3118('<(3)i')
ValueError: Unknown PEP 3118 data type specifier '(3)i'

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions