WIP: MAINT: avoid numpy internals in {sg}et_array_base

[mattip]

In the discussion of issue #2498, it seems cython uses direct access of ndarray structure base attribute in set_array_base and get_array_base. This bypasses much of the NumPy error checks, and can lead to violating assumptions NumPy makes as to the correctness of the base attribute.

This PR refactors the functions to use NumPy APIs. I am not sure it is correct, I could find no tests for the functionality, and truthfully struggle to see when these functions could be properly used. I would prefer to deprecate them.

NumPy uses the base attribute internally when

• creating a view by sharing the data attribute of both self and base, or
• in creating temporary buffer data in ufuncs via writeback semantics.

The simple code in these functions does none of the data attribute manipulations done by those two internal uses.

[mattip]

Any ideas why the c++ tests are failing but the c are passing?

[scoder]

The tests fail with this error:

convolve2.cpp: In function "void __pyx_f_5numpy_set_array_base(PyObject*, PyObject*)":
convolve2.cpp:4563:94: error: cannot convert "PyObject*" {aka "_object*"} to "PyArrayObject*" {aka "tagPyArrayObject_fields*"} in argument passing
   (void)(PyArray_SetBaseObject(__pyx_v_arr, __pyx_v_base));

C isn't as strict as C++ here.

[scoder]

No need to cast here, since this is clearly using Python object access, which returns an … object! :)
But why isn't this using PyArray_BASE()? (Probably not expected to occur in performance critical code …)

[mattip]

I wanted it to go through the python-level attribute access, maybe that is extreme. I will redo this with PyArray_BASE.

[scoder]

The first argument is an ndarray.

[mattip]

fixed

[scoder]

Just keep ndarray arr in the function signature of set_array_base(). That's what the signature was, and that's also what's needed for this call.

[mattip]

ok

[mattip]

Thanks for the help. I admit not knowing how the failing tests are generated, so I couldn't run them before pushing. I also left some lingering questions around the use of PyArray_BASE and refcounts.

[mattip]

Reading the documentation helps, now running tests locally

[scoder]

The NumPy documentation isn't clear here, but I assume that PyArray_BASE() is just a macro that returns a borrowed reference? If so, then the correct return type here is PyObject * and not a normally refcounted object. You'll then have to cast it to <object> on use, which will turn it into an owned reference.

[mattip]

yes, it's just a macro. Fixing to old logic

[scoder]

I guess so, as it was done before.

[scoder]

See my comment regarding the macro signature.

[mattip]

It seems the clang failures are unrelated