MPS / Apple-Silicon GPU acceleration pathway (gated on float32 stability)

[neuromechanist]

Research outcome: .context/mps_pathways.md.

Bottom line

Apple-Silicon GPUs have no FP64 hardware -- PyTorch MPS and MLX both support float64 only on CPU (confirmed mid-2026). AMICATorchNG computes float64 for parity, so the parity path can never run on an Apple GPU. Every MPS pathway is gated on a numerically stable float32 (or mixed-precision) AMICA.

Pathways

• A (enabler): float32 stabilization via Kahan/compensated summation + mixed precision. The sufficient-stat accumulation sums ~30504 samples in float32 (a classic precision sink and the likely divergence source Stabilize float32 AMICATorchNG for the GPU fast path #70 missed). Kahan gives near-float64 accuracy independent of N, ~2-3x op cost. Prerequisite for everything.
• B: large-workload regime. MPS lost on 32 channels because of fixed per-op dispatch overhead, which amortizes on big tensors. After A, sweep channel count / block / n_models to find where MPS beats CPU (likely high-channel montages, not 32).
• C: MLX port (2-3x over PyTorch MPS, Apple-native compiler/fusion) -- higher ceiling, full backend rewrite, still needs A. v2 option.
• D: FP64 emulation (metal-float64) -- DEAD END (custom-Metal-only, no transcendentals, 18-32x penalty).

Recommendation

Gated on Pathway A. Sequence: Kahan+mixed-precision float32 -> dimension-sweep MPS benchmark (B) -> MLX (C) later. Until then float64-CUDA (4.5x, bit-safe) is the production GPU path. Prioritize by how many users run Apple Silicon with large montages.

Depends on the parked float32 work (#70).