AVX-512 parallel dispatch threshold miscalibration causes throughput valley at N=10k for Laplace and Uniform on Zen4

[OldCrow]

Summary

On Zen4 / AVX-512 (Windows), the scipy comparison benchmark shows a pronounced throughput valley at N=10k for at least two distributions where larger and smaller N both perform significantly better. This pattern is characteristic of premature parallel dispatch: the thread pool is invoked at a batch size where synchronisation overhead dominates the gain.

Resolution

All items fully resolved in perf/dispatch-threshold-recalibration (PR #53).

Threshold changes applied

Distribution / Op	Old threshold	New threshold	Evidence
Laplace PDF	64	35,000	Profiler floor artefact; 3-pass sweep
Laplace LogPDF	64	50,000	Severe trough; amortises at N=45-50k
Laplace CDF	1,024	20,000	Same pattern
Uniform PDF	50,000	NEVER	Parallel never recovers; trivial SIMD path
Uniform LogPDF	50,000	NEVER	Same
Uniform CDF	128	NEVER	L2→L3 cache boundary, not dispatch

Gaussian/Exponential items — closed as AMD Precision Boost 2 / TDP artefact

The apparent N=90k cliff in Gaussian PDF/LogPDF and N=40-50k valley in Exponential PDF, observed in loaded-machine fine-grained sweeps, were investigated on a cold machine (idle overnight, fresh from sleep, CPU temp ≤50°C, single open session).

Cold-machine results:

• Gaussian PDF N=90k: 658M/s (loaded run: 186M/s) — no cliff
• Gaussian LogPDF N=90k: 1.7G/s (loaded run: 212M/s) — no cliff
• Exponential PDF N=90k: 539M/s — no sustained valley

Temperature monitoring during the cold-machine run confirmed ≤50°C (far below the ~90°C thermal throttle threshold). CPU utilisation pinned to 100% repeatedly during loaded runs, exhausting the PPT (package power) budget. AMD PB2 then steps the frequency down from boost (~4.5-5.0 GHz) to TDP-limited sustained frequency (~3.0-3.5 GHz) — this is a power constraint, not thermal. The step-down is non-deterministic in timing, which explains why dips appear at different N values (75k, 95k, 110k) in different runs rather than consistently at N=90k.

No threshold changes required for Gaussian PDF/LogPDF (1M/400k) or Exponential PDF (250k). Values are correct. The PB2/TDP mechanism and mitigation are documented in the kAvx512 section header of dispatch_thresholds.h (commit 0c8dae8).

Mitigation for future Zen4 sweeps: use targeted single-distribution --sizes runs to keep sustained CPU load below the PPT threshold.

[OldCrow]

NEON / M1 findings (perf/dispatch-threshold-recalibration)

Same class of problem identified on M1 / NEON via the same pylibstats sweep methodology. Three distributions had genuine dispatch troughs with minimum at N=10k:

Distribution / Op	Old kNeon	New kNeon	Trough	Pre-trough
Laplace PDF	6144 (floor artefact)	35000	268M/s at N=10k	330M/s VECTORIZED
Rayleigh PDF	10000	20000	179M/s at N=10k	214M/s VECTORIZED
LogNormal PDF	10000	25000	180M/s at N=15k	207M/s VECTORIZED

M1-specific finding: L1 cache boundary at N≈5k masks real troughs

M1 L1 data cache is 64KB per performance core. Two-array footprint (input + output doubles) crosses L1→L2 at N≈4096 elements, producing a systematic throughput drop at N≈5k across all operations regardless of threshold. This initially appeared as dispatch troughs in the Gaussian/Exponential/Laplace log_PDF operations — all of which have kNeon threshold=64 (profiler floor, parallel always active). Those N=5k drops are entirely cache-driven; raising those thresholds is not warranted.

Distinguishing heuristic: an L1 cache trough is pinned to N≈5k on M1 regardless of threshold. A genuine dispatch trough minimum shifts to match the threshold value.

Laplace PDF required iterative calibration (clean-entry criterion)

The Laplace PDF oscillatory parallel recovery meant three passes were needed. At each intermediate value (25k, 30k), the first parallel measurement was still below the last VECTORIZED point — indicating the threshold was still too low to amortise GCD overhead. At 35k the parallel entry jumped +20% above the last VECTORIZED point (460M/s vs 383M/s), confirming amortisation. This "clean-entry criterion" is now documented in SIMD_OPTIMIZATION_REFERENCE.md.

All three corrections are in the commit on perf/dispatch-threshold-recalibration; benchmark sweep confirms troughs eliminated and accuracy tables unchanged.