Optimize managed FastTree Sumup to native parity on arm64

[vladimir-aubrecht]

Problem

FastTree builds feature histograms via Sumup, the per-iteration hot loop of tree
training. On x64/x86 this uses the native FastTree library; on arm64 (and any
platform where the native library isn't available) it falls back to the generic
managed IntArray.Sumup, which goes through the IIntArrayForwardIndexer interface
with per-element bounds checks. That fallback is ~1.8x slower than native and
allocates on every call.

Note: the native Sumup (Sumup.h) is itself a plain scalar loop — the only real
SIMD in FastTreeNative is in segment.cpp (one-time segment compression), not in the
per-iteration histogram path. So there is no algorithmic reason managed can't match it.

Change

Add optimized managed Sumup implementations that mirror the native templates
(Sumup.h / SumupNibbles.h / SumupSegment.h) exactly, using fixed pointers and
no bounds checks:

• DenseIntArray: new SumupManagedDense covering 4/8/16/32-bit × weighted/unweighted
  × root (no doc indices)/leaf. Dense8/4/16/32BitIntArray now point their managed
  handler at it instead of the slow base.Sumup.
• SegmentIntArray: new SumupManaged mirroring SumupSegment /
  SumupSegment_noindices for the compressed segment format.

Native remains the default on x64/x86 (UseFastTreeNative is unchanged) — only the
managed fallback path is replaced, so arm64 picks up the fast path automatically.
Because the loops iterate in the same order as native, float accumulation is
bit-identical and existing baselines are unchanged.

Results (Apple M5, arm64)

Real in-repo types (FeatureHistogram.SumupWeighted), Dense8, weighted, 256 bins:

case	N	old fallback	new managed	speedup	identical
root	1,000,000	1.119 ms (894 Melem/s)	0.632 ms (1582 Melem/s)	1.77x	yes
leaf	500,000	0.553 ms (905 Melem/s)	0.363 ms (1378 Melem/s)	1.52x	yes

Native comparison (native Sumup.h scalar loop built for arm64 vs equivalent managed,
N=20M): native 1337 Melem/s, old fallback 746 Melem/s (1.79x slower), new managed
1386 Melem/s (0.96x = parity). Managed allocations per call: 20 B -> 0 B.

Testing

• FastTree/FastForest/FastRank baseline tests in Microsoft.ML.Predictor.Tests pass on
  arm64 (7 passed, 0 failed; the managed path is what runs there, compared against
  native-generated baselines -> confirms numerical identity).
• Standalone verification of the two non-trivial decoders: segment bit-unpacking
  (2000 random trials, sequential + indexed) and the 4-bit nibble formula
  (5000 trials) — 0 mismatches.

Notes for reviewers

• This does not change x64/x86 behavior; native stays the default there.
• UseFastTreeNative is intentionally left unchanged. If we later want to drop the
  native library entirely, this managed path is now fast enough to be the default on
  all architectures — but that is a separate decision.

Fixes #

[vladimir-aubrecht]

@dotnet-policy-service agree company="Microsoft"

[Copilot]

Pull request overview

This PR improves FastTree training performance on arm64 (and other platforms without the native FastTree library) by replacing the slow managed IntArray.Sumup fallback with optimized, bounds-check-free managed implementations that mirror the native scalar Sumup templates to preserve bit-identical histogram accumulation.

Changes:

• Added an optimized managed dense Sumup implementation (SumupManagedDense) and routed Dense 4/8/16/32-bit arrays to use it when native is unavailable.
• Added an optimized managed segment Sumup implementation (SegmentIntArray.SumupManaged) mirroring native segment decoding for both sequential (root) and indexed (leaf) cases.
• Updated handler selection so managed fallbacks use the new optimized implementations instead of base.Sumup.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/Microsoft.ML.FastTree/Dataset/SegmentIntArray.cs	Switches managed fallback to a new pointer-based segment Sumup implementation matching native decoding/accumulation order.
src/Microsoft.ML.FastTree/Dataset/DenseIntArray.cs	Introduces pointer-based dense Sumup implementation and wires Dense 4/8/16/32-bit arrays to use it on non-native platforms.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.