Potential post-2.0 atomic improvements

[ronawho]

Meta issue to capture a list of improvements to atomics that we're considering, but aren't part of Chapel 2.0. https://github.com/Cray/chapel-private/issues/3730 may have additional historical context.

API:

Stabilize compareAndSwap

• Stabilize atomic compareAndSwap #22870

Support additional operations (min/max, maybe mult, maybe inc/dec)

• Support atomic min/max #22867
• Support atomic multiply? #22868
• Support atomic increment/decrement? #22869

Non-atomic updates:

For performance reasons, there's interest in being able to mix atomic and non-atomic updates to the same data:

• Support non-atomic updates on atomics #22875

Locality Optimizations

When accessed locally, network atomics tend to have a pretty high overhead compared to processor atomics. This is because in order to be coherent all operations have to go to the NIC, which involves going across the PCI bus. If you're mostly going to access atomics locally you likely want to user processor atomics.

This is discussed more in #10551. Today we just have an undocumented chpl__processorAtomicType to force processor atomics, but I think we want some sort of user facing version to indicate if an atomic will:

• Only be accessed locally
• Mostly be accessed locally
• Mostly be accessed remotely

And this would allow us to select between pure processor atomics, processor atomics wrapped with on-stmts, and network atomics.

This is mostly about selecting implementation on a global level, but you could imagine wanting finer grain access (I know this update is always local) or wanting to do phases, so this might relate more to #22875 than I was originally thinking.

Operator overloads

Today we only support methods like .add() on atomics and don't support +=. We did this originally on the basis that atomics are special and it should be more obvious in source code that atomic operations are happening. I think most of us are on board with operators today, but it isn't a breaking change so we didn't prioritize for 2.0. The following issues discuss this:

• Should we support convenience operators for methods on atomics #16238
• Should we support direct reads/writes to atomics? #16237
• Should we support operators (like + and + reduce) on atomics? #16239
• Utility operators for atomic variables #8847
• Support reductions over arrays of atomics #12549

Atomic Objects

Today we only support atomics on bool, and all sizes of int, uint, and real. Notably missing from this is support for objects/classes, which are important for creating lock-free data structures in other languages. We don't support this today because Chapel classes can be "wide" (128-bits representing the locale that owns them and the address on that locale.) Not all CPUs (and very few networks) support 128-bit atomic operations so you either need to pack pointers to support atomics, protect access with locks, or have some out-of-band data structure to support this. There's prototype work for packed atomic objects in https://chapel-lang.org/docs/modules/packages/AtomicObjects.html, but it has limitations (API, scale, processor type, configuration.)

Other issues that discuss this:

• Support for atomic class variables #18461
• GlobalAtomicObject - Scalable Atomic Operations using Descriptor Tables #6663
• API Design for new 'atomic_object' type #8710
• GlobalAtomicObject - Scalable Atomic Operations using Descriptor Tables #6717
• [W.I.P] Atomic Object using CMPXCHG16B #8643

Note that most of these discussions were for unmanaged objects and this gets even more complicated for shared objects, which in addition to locale/address have the reference count. C++20 added support for atomic shared_ptr (https://en.cppreference.com/w/cpp/memory/shared_ptr/atomic2), which "may or may not be lock free".