Bumping C++ baseline following Python 3.10 deprecation

[ianna]

Hi everyone,

Following up on our roadmap to drop Python 3.10 support, we have an ideal window to upgrade our core kernel baseline (awkward-cpp) from C++17 to a newer standard.

Given our project's technical direction -- specifically implementing layout support for float16 and complex256 while expanding our use of Python cuda.compute and Numba-MLIR -- I propose migrating our backend baseline to C++20 or C++23.

Why C++23 perfectly fits our CUDA & Numba-MLIR Ecosystem

• Modern toolchains and NVCC now fully support core language features up through C++23, meaning we aren't compiler-restricted from targeting either standard.
• Native Extended Floats (The C++23 Advantage): If we choose C++23, we gain access to <stdfloat>. This standardizes types like std::float16_t and std::float128_t, allowing us to natively build true 256-bit complex numbers (std::complex<std::float128_t>) that align with NumPy arrays without writing custom struct hacks.

Questions for the Maintainers

• Are there immediate downstream packaging limits or CI/CD constraints that restrict us to C++20, or are we clear to move our cibuildwheel targets to support C++23?
• If we go with C++23, should we immediately adopt native <stdfloat> types, or should we still write unified type wrappers to guarantee cross-platform layout compatibility with NumPy?
• Which specific modern standard features (Concepts, Ranges, Monadic Optionals) are you most eager to utilize in our C++ code?

I'd love to hear the team's thoughts on coordinating this language upgrade with our evolving GPU backend!

[Kaustav5505g]

Downstream Packaging & Toolchain Constraints (cibuildwheel & CUDA)
GCC/Clang vs. NVCC Support: While GCC 13+ and Clang 16+ support , NVCC compatibility depends on the host compiler standard driver. If targeted host compilers in older manylinux base images (like manylinux2014 / GCC 10 or 11) don't fully support C++23 , we may need to update the cibuildwheel container matrix to manylinux_2_28 (GCC 12+) or manylinux_2_34.

Apple Clang Limitation: Xcode's Apple Clang historically lags behind upstream LLVM in C++23 header implementations. macOS wheel builds on GitHub Actions runners should be verified before locking in C++23 as a strict requirement.

2. Native vs. NumPy ABI Layout Compatibility
   Standardized Storage vs. ABI Alignment: While std::complexstd::float128_t natively gives us 256-bit complex types, C++23 types are optional implementations (STDCPP_FLOAT16_T / STDCPP_FLOAT128_T macro guards are required).

Recommended Architecture: We should use thin layout-wrapped types backed by standard C++23 types when feature-guarded, fallback-aliased to NumPy-compatible structures otherwise:

C++
#include
#include

namespace awkward::types {
#if defined(STDCPP_FLOAT16_T)
using f16 = std::float16_t;
#else
using f16 = awkward::fallback_float16; // NumPy-compatible bit struct
#endif

#if defined(STDCPP_FLOAT128_T)
using c256 = std::complexstd::float128_t;
#else
using c256 = awkward::complex256_fallback;
#endif
}
This gives us native C++23 speed on supporting compilers while maintaining exact NumPy C-API/PyCapsule buffer memory layout alignment across all platforms.

3. High-Value Standard Features for awkward-cpp
   std::expected / Monadic Optionals (C++23): Exceptional for zero-cost error propagation across Numba/C-ABI boundaries, removing the need for out-parameter error codes in kernel routines.

Concepts & Constrainable Kernels (C++20/C++23): Greatly simplifies meta-programming when dispatching index-types (int32 vs int64) and array payloads, drastically reducing SFINAE bloat and improving compiler error diagnostics.

Explicit Object Parameters / Deduplicating this (C++23): Simplifies CRTP patterns in index/layout visitors and reduces template boilerplate across array transformation nodes.

Bottom Line: Moving to C++23 makes complete sense for and monadic error handling. I propose bumping our cibuildwheel images to manylinux_2_28 and implementing feature-flagged wrappers around to guarantee NumPy C-ABI layout parity across all platform variants.

[ianna]

Reference #4467 that implements C++23 migration.

[Kaustav5505g]

Hey ianna, awesome to see this moving forward into a PR so quickly! Glad the fallback wrapper pattern and moving cibuildwheel to manylinux_2_28 worked out smoothly for the implementation. Let me know if you need any extra eyes on testing the extended-precision layout behavior.

[ianna]

Thanks! Extra eyes are always appreciated!

[Kaustav5505g]

Glad this helped kick off the migration! Feel free to mark this as the answer if you want to close out the discussion.