Numpy 2.x incompatibilities

[djhoese]

See #2345 where I discovered when building against numpy 2.0 that there were some compilation errors. Primarily:

       gcc -pthread -B /usr/share/miniconda3/envs/test-environment/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /usr/share/miniconda3/envs/test-environment/include -fPIC -O2 -isystem /usr/share/miniconda3/envs/test-environment/include -fPIC -DH5_USE_110_API -DH5Rdereference_vers=2 -DNPY_NO_DEPRECATED_API=0 -Ih5py -I/tmp/pip-req-build-3skh7xs0/lzf -I/tmp/pip-req-build-3skh7xs0/lzf/lzf -I/usr/share/miniconda3/envs/test-environment/lib/python3.11/site-packages/numpy/_core/include -I/usr/share/miniconda3/envs/test-environment/include/python3.11 -c /tmp/pip-req-build-3skh7xs0/h5py/h5t.c -o build/temp.linux-x86_64-cpython-311/tmp/pip-req-build-3skh7xs0/h5py/h5t.o
      In file included from /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:77:
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c: In function ‘__pyx_f_4h5py_3h5t__c_complex’:
      /tmp/pip-req-build-3skh7xs0/h5py/api_compat.h:38:31: error: request for member ‘real’ in something not a structure or union
         38 | #define h5py_offset_n64_real (HOFFSET(npy_complex64, real))
            |                               ^~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:26702:21: note: in expansion of macro ‘h5py_offset_n64_real’
      26702 |     __pyx_v_off_r = h5py_offset_n64_real;
            |                     ^~~~~~~~~~~~~~~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/api_compat.h:39:31: error: request for member ‘imag’ in something not a structure or union
         39 | #define h5py_offset_n64_imag (HOFFSET(npy_complex64, imag))
            |                               ^~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:26711:21: note: in expansion of macro ‘h5py_offset_n64_imag’
      26711 |     __pyx_v_off_i = h5py_offset_n64_imag;
            |                     ^~~~~~~~~~~~~~~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/api_compat.h:40:32: error: request for member ‘real’ in something not a structure or union
         40 | #define h5py_offset_n128_real (HOFFSET(npy_complex128, real))
            |                                ^~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:26798:21: note: in expansion of macro ‘h5py_offset_n128_real’
      26798 |     __pyx_v_off_r = h5py_offset_n128_real;
            |                     ^~~~~~~~~~~~~~~~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/api_compat.h:41:32: error: request for member ‘imag’ in something not a structure or union
         41 | #define h5py_offset_n128_imag (HOFFSET(npy_complex128, imag))
            |                                ^~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:26807:21: note: in expansion of macro ‘h5py_offset_n128_imag’
      26807 |     __pyx_v_off_i = h5py_offset_n128_imag;
            |                     ^~~~~~~~~~~~~~~~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/api_compat.h:44:32: error: request for member ‘real’ in something not a structure or union
         44 | #define h5py_offset_n256_real (HOFFSET(npy_complex256, real))
            |                                ^~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:26894:21: note: in expansion of macro ‘h5py_offset_n256_real’
      26894 |     __pyx_v_off_r = h5py_offset_n256_real;
            |                     ^~~~~~~~~~~~~~~~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/api_compat.h:45:32: error: request for member ‘imag’ in something not a structure or union
         45 | #define h5py_offset_n256_imag (HOFFSET(npy_complex256, imag))
            |                                ^~~~~~~
      /tmp/pip-req-build-3skh7xs0/h5py/h5t.c:26903:21: note: in expansion of macro ‘h5py_offset_n256_imag’
      26903 |     __pyx_v_off_i = h5py_offset_n256_imag;
            |                     ^~~~~~~~~~~~~~~~~~~~~
      error: command '/usr/bin/gcc' failed with exit code 1

This is caused by numpy's complex types no longer being structs, but regular float/double types now:

numpy/numpy#24085

This h5py usage is here:

h5py/h5py/api_compat.h

Lines 38 to 41 in de331ae

	#define h5py_offset_n64_real (HOFFSET(npy_complex64, real))
	#define h5py_offset_n64_imag (HOFFSET(npy_complex64, imag))
	#define h5py_offset_n128_real (HOFFSET(npy_complex128, real))
	#define h5py_offset_n128_imag (HOFFSET(npy_complex128, imag))

I'm mostly making this issue for record keeping, but also to link numpy maintainers to for help.

[tacaswell]

From 48e0d4ca462b60fcd89e795c77d7e4056921f9b4 Mon Sep 17 00:00:00 2001
From: Thomas A Caswell <tcaswell@bnl.gov>
Date: Thu, 23 Nov 2023 16:04:35 -0500
Subject: [PATCH] MNT: account for changes to complex types in numpy2

---
 h5py/api_compat.h | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/h5py/api_compat.h b/h5py/api_compat.h
index 52917f4d..ded997ec 100644
--- a/h5py/api_compat.h
+++ b/h5py/api_compat.h
@@ -35,14 +35,14 @@ typedef void *PyMPI_MPI_Message;
 #define h5py_size_n256 (sizeof(npy_complex256))
 #endif

-#define h5py_offset_n64_real (HOFFSET(npy_complex64, real))
-#define h5py_offset_n64_imag (HOFFSET(npy_complex64, imag))
-#define h5py_offset_n128_real (HOFFSET(npy_complex128, real))
-#define h5py_offset_n128_imag (HOFFSET(npy_complex128, imag))
+#define h5py_offset_n64_real 0
+#define h5py_offset_n64_imag h5py_size_n64 / 2
+#define h5py_offset_n128_real 0
+#define h5py_offset_n128_imag h5py_size_n128 / 2

 #ifdef NPY_COMPLEX256
-#define h5py_offset_n256_real (HOFFSET(npy_complex256, real))
-#define h5py_offset_n256_imag (HOFFSET(npy_complex256, imag))
+#define h5py_offset_n256_real 0
+#define h5py_offset_n256_imag h5py_size_n256 / 2
 #endif

 #endif
--
2.42.1

I have been carrying this patch locally, but I am pretty sure it is not the correct fix.

[djhoese]

This should be fixed now in #2345. In the numpy issue it was suggested to use sizeof the complex types sub-type. So for a 64-bit complex number that would be a 32-bit float float.

[tacaswell]

I think this was closed by #2345 as well.