Switch to using Python SharedMemory

In order to resolve portability challenges with use of shared memory, switch to using multiprocessing.shared_memory.SharedMemory from the standard library. This fixes tests on MacOS. One workaround is still needed to manually disable resource tracking, but that monkey patch will not be needed after python-3.13.0. One possible concern with this approach is that the resource_tracker code used by this python package spawns a helper process. Historically, some MPI implementations were very unhappy with this. However, I tested this with OpenMPI on Linux and MacOS, and also with Cray MPICH, running across multiple nodes. No problems observed so far.

Fix catch of unit test failures

• Unit test failures were not triggering a non-zero return to the calling shell. Now fixed.
• Moved the pre-deletion of the shared memory segment until after all processes have wrapped the buffer in a numpy array, ensuring that the buffer is not released too early. This fixes a failure on macos.

Move to sysv_ipc

The posix_ipc module is not available on conda-forge. This release moves the codebase to using sysv_ipc, which is available.

Fix python version in deploy action

• Fix a typo

Small updates to tests and CI workflows

• Bump python versions for tests and requirements.
• Fix use of dtype np.int_ in tests.
• Use concurrency in github actions rather than cancel-workflow action.
• Update versioneer for python 3.12 compatibility.

Use Numpy Arrays for Single Process per Node

Even in the MPI case, if there is only one process in the node shared communicator, then use a simple numpy array. This reduces the total number of shared memory segments, which is limited by the kernel in terms of maximum number of open files (since shared memory segments appear as files to userspace).

Use POSIX shared memory instead of MPI shared windows

This converts the MPIShared class to use POSIX shared memory underneath. We had been using MPI shared windows for this even though we are managing write access to this memory ourselves. Most MPI implementations have a limit on the number of global shared memory buffers that can be allocated and this is insufficient for many cases. This new code is limited only by the number of shared memory segments per node supported by the kernel. The unit tests were run successfully up to 8192 processes.

Fix the array interface

This just adds pass-through methods to the underlying data view.

Add array interface to MPIShared

This defines the __array__() method to the MPIShared class, which exposes the data member when wrapping in a numpy array.

Fix shared memory allocation on some MPI implementations

On some MPI implementations, allocating zero bytes of shared memory on some processes produces an error. Allocate a small dummy buffer on those processes as a workaround.