numpy C API notes

First, my current, general understanding of application binary interfaces (ABIs) and application programmatic interfaces (APIs):

• You can maintain ABI compatibility but break API compatibility (See the note here). For instance, you can have a function that always accepts an integer and returns a float. If you, however, change what that function does for different integer arguments, you have changed the API without changing the ABI. You can build, link, and execute correctly, but your outputs will change.
• You can also change the ABI but not the API. For instance, if a function returns a numpy array, then the ABI might be different from numpy 1.X to 2.Y if the numpy array data structure has changed or is compiled differently despite the fact that the function's behavior (and therefore API) is unaltered. Empirical testing with surmise indeed suggests that this happened between 1.X and 2.Y, which is known to be an ABI-breaking major transition.
• If, however, you change the API significantly by, for example, adding a new argument to a function or a data structure or altering the type of an argument or return variable, then the ABI must also change.

Some numpy packaging details

• the numpy package includes both a Python and a C interface. The latter is available for direct use from C code or indirectly through the use of Cython or similar tools. If using Cython, then numpy is a build-time dependency of the package and it is important to understand potential ABI and API compatibility issues.

NumPy’s ABI is forward but not backward compatible. This means: binaries compiled against a given target version of NumPy’s C API will still run correctly with newer NumPy versions, but not with older versions.

• Strangely, the numpy docs make a statement that implies that the numpy C interface is backward compatible starting at 1.25

Before NumPy 1.25, the NumPy C-API was not exposed in a backwards compatible way by default. This means that when compiling with a NumPy version earlier than 1.25 you have to compile with the oldest version you wish to support.

• Based on the same section, I believe that for numpy >= 1.25

By default, NumPy will expose an API that is backwards compatible with the oldest NumPy version that supports the currently oldest compatible Python version.

• Therefore, since Python 3.9 is the oldest currently supported Python and numpy 1.19 was the first numpy version to support Python 3.9, then all numpy versions >= 1.25 that support Python 3.9 will be backward compatible with all older numpy versions back to and including 1.19.
• Numpy docs explain that while numpy uses the PEP440 major.minor.bugfix version scheme, the scheme is not semantic versioning.
• numpy has a scheme for managing when and how to break backward compatibility.
• A change in major version will most likely imply ABI breakage. They acknowledge that some ABI breakage might have occurred in the past for minor versions.
• Minor versions can include changes to the interface including adding new features and removing deprecated code. However, backwards-incompatible interface changes require the use of deprecation warnings in at least the two preceding releases.
• Numpy 2 docs indicate both API and ABI breakages.
• At present, wheels built with 1.X numpy will not work with 2.Y numpy installations. However, wheels built with 2.Y numpy will work with both 1.X and 2.Y versions.
• In principle, it appears that a package that uses Cython+numpy can work with numpy >=1.25 so long as they compile wheels with >=2.0.0 and the numpy functionality they use are not deprecated at a 2.Y minor release. This would require, however, that the numpy ABI be somewhat backward-compatible contrary to what the numpy documentation says. If you want to adhere 100% to numpy's statements, then you should compile wheels with 2.0.0 since that is stated to be backward compatible with 1.X and should work with > 2.0.0 since the ABI is guaranteed to be forward ABI compatible but not necessarily backward compatible. This would work so long as the numpy API used by surmise is not changed at a 2.Y minor revision. Note that such a change might also force a change of ABI, in which case the ABI is not forward compatible either.