Skip to content

SyclReproSum 1.1.0

Choose a tag to compare

@lilohuang lilohuang released this 12 Jul 07:43
· 15 commits to main since this release

SyclReproSum 1.1.0 adds bit-reproducible cumulative sums and strengthens the
floating-point, capacity, device-reduction, and resource-safety guarantees of
the scalar summation API. The library remains header-only, C++17, and based on
the Ahrens-Demmel-Nguyen (ADN) binned floating-point format, with binned
primitives adapted from the ReproBLAS reference implementation.

Highlights

  • New adn::cumsum API for reproducible inclusive cumulative sums of float
    and double values on SYCL CPU and GPU devices.
  • Bit-identical cumulative results across runs, thread schedules, supported
    work-group sizes, and validated devices for a fixed input sequence.
  • Exact in-place cumulative sums with input == output.
  • Independent support for plain host pointers and device, shared, or host USM
    pointers for both cumulative-sum input and output.
  • A hierarchical binned scan that stores binned state only for tile totals and
    tile prefixes, rather than one full accumulator per output element.
  • Stronger fail-closed environment validation for both adn::sum and
    adn::cumsum, including effective device arithmetic probes.
  • Explicit accumulator-capacity checks through the public
    adn::max_reproducible_count<T> constant.
  • Final scalar reduction and conversion now remain on the device with explicit
    event ordering, making the result independent of hostile host floating-point
    modes.
  • Exception-safe ownership and cleanup for all internal USM allocations.
  • Version macros now report 1.1.0.

New reproducible cumulative-sum API

adn::cumsum computes an inclusive reproducible prefix sum:

#include <sycl/sycl.hpp>
#include <vector>
#include "repro_sum.hpp"

sycl::queue q{sycl::gpu_selector_v};

std::vector<double> input = {1.0, 2.0, 3.0, 4.0};
std::vector<double> output(input.size());

adn::cumsum(q, input.data(), output.data(), input.size());
// output is {1.0, 3.0, 6.0, 10.0}

The template parameters match adn::sum:

adn::cumsum<6>(q, input, output, count);       // More folds
adn::cumsum<3, 512>(q, input, output, count); // Custom work-group size

For every valid index i, output[i] is the ADN reproducible sum of
input[0] through input[i]. The call is synchronous: when it returns, a
plain host output buffer has been copied back and is ready to use.

Exact in-place operation is supported:

adn::cumsum(q, data, data, count);

Partially overlapping ranges other than exact input == output are not
supported.

Cumulative-sum reproducibility model

Input order defines which values belong to each prefix. For a fixed input
sequence, every prefix result is bit-identical across runs, supported
work-group sizes, thread schedules, and validated CPU or GPU devices.

Shuffling an input generally changes the intermediate prefixes because it
changes prefix membership. The last cumulative output is special: it contains
all input values and retains the full order-independent guarantee of
adn::sum. In other words:

  • Every output is reproducible for a fixed sequence.
  • The last output is also independent of input permutation.
  • output[count - 1] is bit-identical to adn::sum(q, input, count).

NaNs and infinities follow the same deterministic rules as scalar sums. Any
NaN, or a prefix containing both positive and negative infinity, produces the
fixed positive canonical quiet NaN for that and all following prefixes:

  • float: 0x7FC00000
  • double: 0x7FF8000000000000

Subnormal values, signed zero, overflow, and gradual underflow are handled by
the same binned primitives used by adn::sum.

How the cumulative scan works

adn::cumsum is a hierarchical scan that keeps intermediate totals in the ADN
binned format:

  1. Contiguous input tiles are reduced to reproducible binned totals.
  2. Tile totals undergo a recursive exclusive scan while remaining in binned
    form.
  3. A final pass replays each tile from its incoming binned accumulator and
    converts every inclusive prefix to float or double.

Only tile totals and tile prefixes require temporary global storage. The
implementation does not materialize a complete K-fold binned accumulator for
every output element.

Pointer handling

The cumulative-sum input and output pointer types are detected independently
at runtime with sycl::get_pointer_type:

Pointer source Input behavior Output behavior
Device USM Used directly Used directly
Shared USM Used directly Used directly
Host USM Used directly Used directly
Plain host pointer Copied to temporary device storage Computed in temporary device storage and copied back

This supports combinations such as a plain host input with device output or a
device input with plain host output. "Used directly" means no library-side
copy; the SYCL runtime may still migrate or otherwise manage USM storage.

Stronger scalar-sum guarantees

This release also hardens adn::sum without changing its public call syntax.

Device-only final reduction

Per-work-group accumulators are now merged and converted by a final device
work-group with explicit SYCL event dependencies. Host-side floating-point
rounding, denormal, and evaluation modes therefore cannot affect the final
result.

Explicit capacity enforcement

The exact binned carry representation has a finite conservative capacity
inherited from ReproBLAS:

Type Maximum reproducible input count
float 4,294,966,784
double 9,223,372,036,854,773,760

The limits are available as adn::max_reproducible_count<float> and
adn::max_reproducible_count<double>. Both adn::sum and adn::cumsum reject
larger inputs with std::length_error before pointer inspection or allocation.
adn::cumsum also rejects a byte-size overflow and requires non-null pointers
when the count is nonzero.

Splitting a larger input into scalar chunks and adding the rounded chunk
results does not preserve the library's reproducibility guarantee.

Strict floating-point requirements

The header continues to reject fast-math and equivalent unsafe compilation
modes. Version 1.1.0 additionally:

  • Verifies the required IEEE 754 binary32 and binary64 layouts at compile time.
  • Requires device expressions to evaluate in their declared type.
  • Explicitly disables reassociation, reciprocal transformations, and FP
    contraction in accumulator arithmetic where supported.
  • Checks the device's advertised round-to-nearest, denormal, Inf/NaN, and fp64
    capabilities.
  • Executes runtime-loaded device probes for effective rounding behavior,
    gradual underflow, and signed-zero preservation.

Both float and double operations require device fp64 support because final
float conversion accumulates in double precision.

Successful validation is cached per backend, device, and floating-point type
and shared safely across host threads. Root-enumerated devices use the fixed
cache; dynamically created sub-devices are validated on each use. Applications
can preflight explicitly with:

adn::validate_environment<float>(q);
adn::validate_environment<double>(q);

Validation throws std::runtime_error when the library cannot establish the
required arithmetic guarantees.

Exception-safe USM cleanup

Internal shared and device USM allocations now use context-aware RAII owners.
If allocation, submission, copying, or synchronization throws, owned storage
is released without throwing from cleanup. When a later queue submission
fails after earlier work was submitted, cleanup waits for pending work before
releasing the associated USM allocation.

Accuracy and configuration

The default remains K=3 folds and a work-group size of 256. Supported
work-group sizes are powers of two from 2 through 1024, subject to the target
device's work-group and local-memory limits. Increasing K trades performance
and storage for additional low-order coverage.

Type Bin width Maximum K Dominant absolute-error scale at K=3
double 40 bits 52 approximately N * 2^-80 * max abs input
float 13 bits 21 approximately N * 2^-26 * max abs input

The error scale is absolute rather than relative. Severe cancellation can make
the error large relative to the exact final sum. The result is reproducible,
but it is not an arbitrary-precision exact sum.

Performance

The included benchmarks process 100 million elements. Each value is the mean
of three benchmark invocations, and each invocation averages ten timed runs
after one warmup. Throughput is measured in billions of elements per second.

Cumulative-sum throughput

The baseline is oneDPL inclusive_scan using a standard device policy and
std::plus on the same device-USM input and output. The baseline is not
reproducible and is provided only as a throughput comparison.

Device Type adn::cumsum oneDPL baseline Slowdown
NVIDIA GB10 via CUDA double 2.364 GElements/s 7.606 GElements/s 3.2x
NVIDIA GB10 via CUDA float 3.413 GElements/s 14.980 GElements/s 4.4x
NVIDIA RTX PRO 4500 Blackwell via CUDA double 4.855 GElements/s 21.383 GElements/s 4.4x
NVIDIA RTX PRO 4500 Blackwell via CUDA float 6.696 GElements/s 40.480 GElements/s 6.0x
Intel Graphics [0x7d67] via Level Zero double 0.167 GElements/s 0.610 GElements/s 3.7x
Intel Graphics [0x7d67] via Level Zero float 0.331 GElements/s 2.445 GElements/s 7.4x
Intel Core Ultra 7 265 CPU via OpenCL double 0.118 GElements/s 1.490 GElements/s 12.6x
Intel Core Ultra 7 265 CPU via OpenCL float 0.138 GElements/s 2.536 GElements/s 18.4x

Scalar-sum throughput

For completeness, scalar-sum results remain:

Device Type adn::sum Plain sycl::reduction Slowdown
NVIDIA GB10 via CUDA double 26.3 GElements/s 31.7 GElements/s 1.2x
NVIDIA GB10 via CUDA float 55.5 GElements/s 62.9 GElements/s 1.1x
NVIDIA RTX PRO 4500 Blackwell via CUDA double 48.7 GElements/s 77.0 GElements/s 1.6x
NVIDIA RTX PRO 4500 Blackwell via CUDA float 121.2 GElements/s 124.5 GElements/s 1.0x
Intel Graphics [0x7d67] via Level Zero double 1.7 GElements/s 6.6 GElements/s 3.8x
Intel Graphics [0x7d67] via Level Zero float 4.1 GElements/s 12.5 GElements/s 3.0x
Intel Core Ultra 7 265 CPU via OpenCL double 0.4 GElements/s 5.8 GElements/s 13.6x
Intel Core Ultra 7 265 CPU via OpenCL float 0.6 GElements/s 7.4 GElements/s 12.3x

Cross-device reproducibility

The one-million-element double wide-range scalar test continues to produce:

0x430FC878C605717F

The same bits, including after independent input shuffles, have been verified
on:

  • NVIDIA GB10 via CUDA
  • NVIDIA RTX PRO 4500 Blackwell via CUDA
  • Intel Graphics [0x7d67] via Level Zero
  • Intel Core Ultra 7 265 CPU via OpenCL

The cumulative-sum cross-device suite also requires every tested prefix to be
bit-identical across the selected devices for a fixed sequence and checks the
last prefix against adn::sum.

Testing and tooling

The expanded Google Test suite now contains 169 correctness cases and four
throughput benchmarks per selected device, plus ten cross-device cases and one
version test. Coverage includes:

  • Prefix-by-prefix references against adn::sum.
  • Multi-level tile scans and large boundary cases.
  • Work-group sizes 64, 128, 256, 512, and 1024.
  • Fold counts 2, 3, 4, 5, 6, 8, and 12.
  • Separate and exact in-place host and USM operation.
  • Bounds guards and mixed pointer types.
  • Repeated-run, cross-work-group-size, and cross-device bit identity.
  • NaN, infinity, subnormal, cancellation, and wide-range inputs.
  • Concurrent environment validation and exception-safe cleanup.
  • Hostile host floating-point modes and rejection of unsafe device modes.

make test-validation now rebuilds the correctness suite under hostile host
floating-point settings, then verifies that unsafe device denormal and
signed-zero configurations are rejected. The formatting workflow is pinned to
clang-format 18 for stable CI behavior. The cumulative-sum benchmark requires
oneDPL headers for its inclusive_scan baseline.

For this release candidate, the normal build completed successfully, all 518
non-benchmark tests passed, the same 518 tests passed under hostile host modes,
and both unsafe-device rejection variants passed.

Compatibility and upgrade notes

  • This is an additive source-compatible release; existing adn::sum calls do
    not require changes.
  • Code that previously ran on an unsafe or insufficiently advertised
    floating-point environment may now receive std::runtime_error instead of a
    result whose reproducibility could not be guaranteed.
  • Inputs beyond adn::max_reproducible_count<T> now fail explicitly with
    std::length_error.
  • No compiled library or link step is required. Replace or update
    repro_sum.hpp and continue compiling as C++17 with SYCL.

Build requirements

  • A C++17 compiler with SYCL support
  • Intel DPC++/LLVM with the required offload target support
  • At least one fp64-capable SYCL CPU or GPU device that passes validation
  • oneDPL headers for the cumulative-sum benchmark baseline
  • The Google Test submodule for the test suite

Typical commands:

make all
make run
make test
make test-validation

make test includes the 100-million-element throughput benchmarks. For a
faster correctness-only run:

./repro_test "--gtest_filter=-*Bench*"

Changes since 1.0.0

  • Add reproducible cumulative sums.
  • Add NVIDIA GB10 cumulative-sum benchmark results.
  • Strengthen reproducible summation guarantees and device validation.
  • Move final scalar reduction and conversion fully onto the device.
  • Enforce conservative accumulator capacities.
  • Make internal USM allocation cleanup exception-safe.
  • Add NVIDIA GB10 scalar benchmark documentation.
  • Pin formatting CI to clang-format 18.
  • Expand contributor, testing, benchmark, and agent documentation.

Full comparison: v1.0.0...v1.1.0

Attribution and license

SyclReproSum is licensed under the BSD 3-Clause License. Portions of the binned
accumulator primitives are adapted from ReproBLAS, Copyright 2016, University
of California. See THIRD_PARTY_NOTICES for the complete attribution and
license terms.