Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow inspection of the data used by PyArray_EquivTypes #245

Merged
merged 5 commits into from
Sep 1, 2017
Merged

allow inspection of the data used by PyArray_EquivTypes #245

merged 5 commits into from
Sep 1, 2017

Conversation

dalleyg
Copy link
Contributor

@dalleyg dalleyg commented Jul 18, 2013

Changes include:

  • Added the PyDataType_HASSUBARRAY macro
  • Added the new NPY_DATETIME and NPY_TIMEDELTA entries to NPY_TYPES
  • Added the "kind" and "subarray" fields to the dtype class

Additional work could be done to expose the datatime metadata, but
non-performance users can already get at that data using the pure
python numpy.datetime_data function.

Author: Gerald Dalley

 - Added the PyDataType_HASSUBARRAY macro
 - Added the new NPY_DATETIME and NPY_TIMEDELTA entries to NPY_TYPES
 - Added the "kind" and "subarray" fields to the dtype class

Additional work could be done to expose the datatime metadata, but
non-performance users can already get at that data using the pure
python numpy.datetime_data function.

Author: Gerald Dalley
@scoder
Copy link
Contributor

scoder commented Jul 18, 2013

As I said, it should be done exactly like the ndarray class, i.e. this (basically) works for me:

ctypedef class numpy.PutTheRightTypeHere [object PyArray_ArrayDescr]:
    cdef tuple shape

However, since I don't have actual code to test this, I can't say what the correct Python type name is that you need to use for the class. Maybe you should discuss this on the cython-users mailing list.

@dalleyg
Copy link
Contributor Author

dalleyg commented Jul 18, 2013

scoder,

What you're suggesting does not work. PyArray_ArrayDescr is a C-only struct. It is not a PyObject. The exact definition in ndarraytypes.h is

typedef struct _arr_descr {
    PyArray_Descr *base;
    PyObject *shape;       /* a tuple */
} PyArray_ArrayDescr;

Note that there is no PyObject_HEAD there. Instead it's a raw C struct that happens to contain two pointers to PyObject instances.

On the Python side, this gets wrapped into a tuple, on-demand:

>>> x = zeros((2,2), dtype=[('a', 'int32'), ('b', 'float64', (3,3))])
>>> d = x.dtype['b']
>>> d.subdtype
(dtype('float64'), (3, 3))
>>> type(d.subdtype)
tuple

The code that dynamically constructs this tuple is in multiarray/descriptor.c:

static PyObject *
arraydescr_subdescr_get(PyArray_Descr *self)
{
    if (!PyDataType_HASSUBARRAY(self)) {
        Py_INCREF(Py_None);
        return Py_None;
    }
    return Py_BuildValue("OO",
            (PyObject *)self->subarray->base, self->subarray->shape);
}

@scoder
Copy link
Contributor

scoder commented Jul 19, 2013

Ah, ok, got it. I had continuously failed to see that "base" is a pointer, not an embedded struct. Sorry, and thanks for insisting.

Structs don't support embedded Python object references, because Cython cannot make any safe assumptions about their reference counting behaviour. So you can only use PyObject* for the shape field. Since that's ugly to use in Cython, would it make sense to provide inlined helper functions in numpy.pxd that provide more or less the functionality of the C implemented descriptior?

Gerald Dalley added 2 commits July 19, 2013 10:15
This makes PyDataType_SHAPE more consistent with the Python API.
@robertwb robertwb force-pushed the master branch 2 times, most recently from dd3e01f to c4c0acc Compare August 21, 2014 06:55
@scoder scoder merged commit 926337d into cython:master Sep 1, 2017
@scoder scoder added this to the 0.27 milestone Sep 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants