Skip to content

v2.3: Rayon-style Parallel Iterators 🦀

Choose a tag to compare

@ashvardanian ashvardanian released this 09 Oct 20:27

This PR introduces Rayon-style parallel iterators for Rust with explicit pool control, adds cache-aligned scratch space primitives, and delivers significant performance improvements across the board.

Rayon-style Parallel Iterator API for Rust

Added ergonomic parallel iterator support with explicit control over thread pools and scheduling:

use fork_union::prelude::*;

let mut pool = fu::spawn(4);
let data: Vec<usize> = (0..1000).collect();

(&data[..])
    .into_par_iter()
    .filter(|&x| x % 2 == 0)
    .map(|x| x * x)
    .with_pool(&mut pool)
    .for_each(|value| println!("Squared even: {}", value));

Key differences from Rayon:

  • Explicit with_pool() instead of global thread pool
  • For work-stealing schedulers: with_schedule(&mut pool, DynamicScheduler)
  • Access to prong metadata: for_each_with_prong(|item, prong| ...)
  • NUMA-aware via distributed pools... transparent to user!

Cache-Aligned Scratch Space

Added CacheAligned<T> wrapper with 128-byte alignment to prevent false sharing in per-thread reductions:

let mut scratch: Vec<CacheAligned<usize>> =
    (0..pool.threads()).map(|_| CacheAligned(0)).collect();

(&data[..])
    .into_par_iter()
    .with_pool(&mut pool)
    .fold_with_scratch(scratch.as_mut_slice(), |acc, value, _prong| {
        acc.0 += *value;
    });
let total: usize = scratch.iter().map(|a| a.0).sum();

Performance Improvements

  • Upgraded C++ optimization flags from -O2 to -O3 -march=native -mtune=native, resolving a major performance gap with Fork Union's Rust benchmarks on macOS.
  • Achieved 20-30% improvement in Rust by avoiding usize pointer casts in favor of typed SyncPtr<T>, enhancing type safety and enabling more LLVM optimizations.

Updated benchmark results for N=128 bodies, 1M iterations:

Machine Rust FU (S) Rust Rayon (S) C++ FU (S) C++ OpenMP (S)
16x Intel SPR 9.8s 🔄 32.1s 8.7s 12.4s
12x Apple M2 11.0s 🔄 1m:07.1s 20.3s 1m:25.9s
96x Graviton 4 10.1s 🔄 1m:35.6s 26.0s 20.8s

Minor

  • Add: CacheAligned for Rust scratch buffers (2f8b6de)
  • Add: Rayon-style parallel iterators (0a66d1d)

Patch

  • Fix: On 32-bit machines avoid exhausting stack in tests (de42b36)
  • Make: OpenMP made optional to avoid "macro redefined" on macOS (b3df7e1)
  • Make: Refresh Rayon & SPR results (0e20c4c)
  • Docs: Rayon-style parallel iterators (ef0ae95)
  • Make: Compile OpenMP on macOS (02983f9)
  • Make: Debug 32-bit i386 with GDB (d7d89d8)
  • Make: -O3 -march=native -mtune=native for C++ (6bea0e5)
  • Docs: Exclude compilation from time-ing (a195fb5)
  • Improve: 25% boost with typed pointers (44cd90b)