make tests run on numpy1.x and numpy2.x

[mattip]

Fixes #6072

Should this also add a numpy2.0b1 test run to CI via the wheels at https://pypi.anaconda.org/scientific-python-nightly-wheels/numpy (i.e. install numpy via this index)?

pip install -i https://pypi.anaconda.org/scientific-python-nightly-wheels/simple numpy

Edit: make it a question

[mattip]

This is better but it does not fix everything. There is some mess since on NumPy1

typedef struct { double real, imag; } npy_cdouble;
typedef struct { float real, imag; } npy_cfloat;
typedef struct { npy_longdouble real, imag; } npy_clongdouble;

but on NumPy2

#if defined(__cplusplus)

typedef struct
{
    double _Val[2];
} npy_cdouble;

typedef struct
{
    float _Val[2];
} npy_cfloat;

typedef struct
{
    long double _Val[2];
} npy_clongdouble;

#else
#include <complex.h>
#if defined(_MSC_VER) && !defined(__INTEL_COMPILER)
typedef _Dcomplex npy_cdouble;
typedef _Fcomplex npy_cfloat;
typedef _Lcomplex npy_clongdouble;
#else /* !defined(_MSC_VER) || defined(__INTEL_COMPILER) */
typedef double _Complex npy_cdouble;
typedef float _Complex npy_cfloat;
typedef longdouble_t _Complex npy_clongdouble;
#endif
#endif

This is making compilation fail since cython assumes .real and .imag can be used on complex types.

[mattip]

There is no simple way to map np.int_ to a C type on windows, since on NumPy1 it is an alias of np.int32 but on NumPy2 it is an alias to np.int64. On non-windows, it is always an alias to np.int64.

[da-woods]

There is no simple way to map np.int_ to a C type on windows, since on NumPy1 it is an alias of np.int32 but on NumPy2 it is an alias to np.int64. On non-windows, it is always an alias to np.int64.

I think dropping np.int_ is reasonable. Or possibly running the test selectively on Numpy2 only.

Not sure what to do about the complex stuff. There probably isn't a sensible way of writing one test for Cython that covers both (because the interface has changed, and what we're really testing in Cython is the interface)

[mattip]

Not sure what to do about the complex stuff.

I think cython should wait for a resolution to numpy/numpy#26029, and adjust the tests appropriately. Clearly arr[1].imag += 1 is wrong, but I am not sure what is right. I would like to have a way to write np_csetimag(np_cimag(arr[1]), np_cimag(arr[1]) + 1) more compactly.

In any case, this PR goes most of the way to fixing the incompatibility in a way that works on both versions, so if CI agrees maybe it could be merged and the rest handled in a subsequent PR.

[mattip]

CI is passing.

[da-woods]

In any case, this PR goes most of the way to fixing the incompatibility in a way that works on both versions, so if CI agrees maybe it could be merged and the rest handled in a subsequent PR.

Yeah I think I'm happy to go along with this.

I'll put off a decision about backporting it to now and just merge it to master I think.

Thanks @mattip

[da-woods]

3.0.x commit e4b1e5e (since it looks like the Python 3.12 tests there use Numpy 2)

[mattip]

3.0.x commit e4b1e5e

I think #6100 is also needed

[da-woods]

3.0.x commit e4b1e5e

I think #6100 is also needed

Yes I think so too - I still need to have a proper look at that

[stratakis]

On numpy 1.26.0 on i686 this test fails with:

Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.12/doctest.py", line 1361, in __run
        exec(compile(example.source, filename, "single",
      File "<doctest numpy_parallel.test_parallel_numpy_arrays[0]>", line 1, in <module>
        test_parallel_numpy_arrays()
      File "tests/run/numpy_parallel.pyx", line 34, in numpy_parallel.test_parallel_numpy_arrays (numpy_parallel.c:4494)
        x = numpy.zeros(10, dtype=numpy.int_)
    ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'