Adopt Python Array API standard

[leofang]

This is a meta-issue tracking the adoption of Python Array API standard:

• Intro: https://data-apis.org/blog/array_api_standard_release/
• API spec: https://data-apis.github.io/array-api/latest/
• NumPy NEP-47: NEP: array API standard adoption (NEP 47) numpy/numpy#18456 (rendered in https://numpy.org/neps/nep-0047-array-api-standard.html)
• Array API repo: https://github.com/data-apis/array-api

In the Data API weekly meetings, it was decided that a reference implementation will be provided to NumPy (see NEP-47), which other implementations can follow and test against. Given that CuPy is IMHO by far the most compliant library with the API standard (probably more compliant than NumPy is, given many design deviations), I think it is much easier if we just wait until the ref implementation is out, and try taking it as is.

A new array object will be added in a separate namespace, delegating real work to cupy.ndarray whenever appropriate. Following NEP-47 I suggest the namespace to be cupy.array_api, but I suppose cupyx.array_api is also a fine choice.
The array object will only have "dunder" methods (names with double underscores), and all APIs for manipulating this object live under the new namespace.

But in addition to the new array object and APIs added to the new namespace, similar to numpy/numpy.ndarray there are some work to be done in the cupy namespace as well as in cupy.ndarray. A list of such examples are copied from NEP-47 for convenience:

• ndarray.__dlpack__
• ndarray.__dlpack_device__
• ndarray.device

In addition, since DLPack is the main data exchange mechanism, we better support

• allocate numpy arrays backed by pinned memory (kDLCPUPinned) (related: Pinned memory allocation returns odd size #3625)

There are still some issues in the standard to be ironed out (ex: linalg related APIs), but I think it's getting close to be completed.

[leofang]

it was decided that a reference implementation will be provided to NumPy

See numpy/numpy#18585.

[leofang]

it was decided that a reference implementation will be provided to NumPy

See numpy/numpy#18585.

@kmaehashi Would CuPy follow exactly what NumPy does and creates a new _array_api namespace, or would you rather follow PyTorch and turn cupy.ndarray directly into an Array API compliant array object? I took the former as granted, but I think I better double check with you guys 😅

[kmaehashi]

We will follow NumPy's design as our goal is to be a drop-in replacement for NumPy 😄

[leofang]

Thanks for confirming @kmaehashi!

[leofang]

Update: NumPy has done its first pass! numpy/numpy#18585