perf: recalibrate parallel dispatch thresholds across all four SIMD tiers (issue #50)#53
Merged
Merged
Conversation
Laplace PDF/LogPDF: 64 → 25000 Laplace CDF: 1024 → 20000 Uniform PDF/LogPDF: 50000 → NEVER Uniform CDF: 128 → 50000 The Laplace PDF/LogPDF thresholds of 64 were at the profiler measurement floor (documented as a warm-pool artefact in the sha-1b564ec bundles). The pylibstats scipy comparison benchmark confirms a throughput trough at N=10k (121M vs 229M at N=5k), with recovery stable from N=20-30k onwards. Uniform PDF/LogPDF thresholds of 50000 matched the observed trough exactly: 1.3G/s collapses to 438M/s at N=50k and does not recover to pre-cliff levels within the 1k-100k sweep. The threshold was already flagged as a bimodal override artefact in the table comments. Uniform CDF threshold of 128 caused an identical cliff at N=10k (833M → 109M). CDF recovery stabilises around N=40-50k. Gaussian PDF/LogPDF and Exponential PDF anomalies (N=90k and N=50k cliffs respectively) are tracked in issue #50 pending strategy_profile validation — those thresholds are NOT changed here. Co-Authored-By: Oz <oz-agent@warp.dev>
Laplace PDF: 25000 → 35000 Laplace LogPDF: 25000 → 50000 Uniform CDF: 50000 → NEVER Laplace PDF/LogPDF: the first pass (25k) moved the dispatch penalty from N=10k to N=25k rather than eliminating it. A targeted 5k-resolution sweep confirmed: - LogPDF at N=25k: 433M → 170M (nearly as bad as original) - Threshold only amortises at N=45-50k; raised to 50k - PDF dip milder; 35k clears it with no visible overhead Uniform CDF: raising from 50k to NEVER revealed the N=50k cliff is an L2→L3 cache boundary (N=45k at 878M fits in L2; N=50k at 800KB two-array footprint does not). The previous 50k threshold was adding parallel overhead on top of a cache miss (463M), making it worse than SIMD-only (552M at N=50k with NEVER). No threshold can fix a cache capacity limit. Co-Authored-By: Oz <oz-agent@warp.dev>
SIMD_OPTIMIZATION_REFERENCE.md: - Detecting threshold miscalibration via external benchmarks: the trough-at-threshold signature, dispatch vs cache boundary effect, profiler floor artefacts, and NEVER threshold criteria. Documented using the v2.0.2 Laplace/Uniform recalibration as worked examples. - Cache hierarchy effects: Zen4 L2/L3/DRAM boundaries and their impact on throughput measurements at different batch sizes. - Cross-architecture accuracy differences: Bessel Tier 1 vs Tier 2 selection, VonMises accuracy implication, scipy version ruling. - Known structural performance ceilings: VonMises CDF (scalar integration), Cauchy CDF (arctan vs incomplete-beta delegation), Binomial (lgamma scalar loop, PMF summation). References issues #47, #48, #51, #52. SIMD_BENCHMARK_RESULTS.md: - v2.0.2 section: machines, peak throughput table, speedup ratios vs scipy at N=100k, full accuracy table with Bessel-tier split for VonMises, and benchmark command reference. Co-Authored-By: Oz <oz-agent@warp.dev>
…DF (issue #50) Laplace PDF: 6144 → 35000 Rayleigh PDF: 10000 → 20000 LogNormal PDF: 10000 → 25000 All three had genuine dispatch troughs (minimum at N=10k, above the M1 L1→L2 boundary at N≈4k) confirmed via the pylibstats scipy_comparison benchmark sweep. Laplace PDF required three passes (25k, 30k, 35k). At 25k and 30k the parallel entry point was still below the VECTORIZED level — threading overhead not yet amortised. At 35k the parallel entry (+20% vs last VECTORIZED point) confirmed amortisation. See clean-entry criterion in SIMD_OPTIMIZATION_REFERENCE.md. LogNormal PDF trough extended to N=15k (deeper than N=10k), motivating the higher threshold of 25k over the naive 20k. Key M1-specific finding: the L1→L2 cache boundary at N≈4096 elements produces a systematic throughput drop at N≈5k across all operations, independent of dispatch threshold. Operations with kNeon threshold=64 (profiler floor) show N=5k drops that are purely cache-driven; no threshold recalibration is warranted for those. Docs: added §L1 data cache boundary on NEON/M1, §Clean-entry criterion for iterative calibration (SIMD_OPTIMIZATION_REFERENCE.md), and NEON threshold recalibration sweep section (SIMD_BENCHMARK_RESULTS.md). Closes #50 for NEON/M1. AVX-512 Laplace/Uniform corrections already encoded in prior commits on this branch. Co-Authored-By: Oz <oz-agent@warp.dev>
…50) Seven kAvx2 corrections confirmed via pylibstats scipy_comparison sweep (three passes: 1k–100k dense, 50k–150k at 10k, 90k–160k at 5k). kAvx2 changes: Uniform CDF: 128 → NEVER (trough N=5k 105M; parallel never recovers within practical range) Gaussian PDF: 50000 → 130000 (trough N=50k–75k; clean entry N=130k 369M > VECTORIZED 347M) Gaussian LogPDF: 64 → 20000 (floor artefact; trough N=7.5k; clean entry N=20k 404M) Exponential PDF: 25000 → 120000 (trough N=25k–35k; clean entry N=120k 280M > VECTORIZED 240M) Exponential LogPDF: 64 → 25000 (floor artefact; trough N=5k 117M; clean entry from N=25k–30k) Exponential CDF: 25000 → 100000 (trough N=25k–30k; clean entry confirmed N=100k 230M > 204M) Laplace LogPDF: 64 → 25000 (floor artefact; trough N=5k 107M; clean entry from N=25k) kAvx derived from kAvx2 (Sandy Bridge/Ivy Bridge; L1/L2 same as kAvx2): ÷2 for transcendental-heavy ops (exp-based: Gaussian/Exponential PDF/CDF) ÷1.5 for polynomial-only ops (Gaussian LogPDF, Exponential LogPDF, Laplace LogPDF): FMA absence causes smaller cost delta than for exp. kNone: comment-only update — corrects 'L1d boundary' claim (actual L1 boundary is N≈2048; 8192 is within L2 range) and notes T3=16384 sits at the L2→L3 boundary, the same failure mode fixed by NEVER in kAvx2/kAvx512 Uniform (issue #50). Numbers unchanged; no profiling hardware available. Co-Authored-By: Oz <oz-agent@warp.dev>
…onential items Cold-machine targeted sweep (2026-07-04) on idle Asus TUF A16 (overnight sleep, single session, CPU temp ≤50°C) confirms the N=90k Gaussian and N=40-50k Exponential cliffs seen in earlier loaded-machine runs are caused by AMD Precision Boost 2 stepping down from boost frequency (~4.5-5 GHz) to TDP-limited sustained frequency (~3.0-3.5 GHz) once PPT budget is exhausted under sustained 100% CPU load. This is a power constraint, not thermal: temperature remained well below the ~90°C throttle threshold throughout. PB2 step-down timing is non-deterministic, so the dip falls at whichever distribution happens to be measured at that moment — confirmed by dips appearing at different N values (75k, 95k, 110k) in the cold-machine run rather than consistently at N=90k. Cold-machine results: Gaussian PDF N=90k: 658M/s (loaded: 186M/s) — no cliff Gaussian LogPDF N=90k: 1.7G/s (loaded: 212M/s) — no cliff Exponential PDF N=90k: 539M/s — no sustained valley kAvx512 Gaussian (1M/400k) and Exponential (250k/400k/250k) thresholds are correct; no numeric changes required. Added section-header note and inline comments documenting the PB2/TDP mechanism and mitigation (use targeted single-distribution --sizes runs for future sweeps). Closes #50 Gaussian/Exponential open items. Co-Authored-By: Oz <oz-agent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Corrects parallel dispatch threshold miscalibrations across kAvx512, kNeon, kAvx2, and kAvx. All corrections were driven by
pylibstats/benchmarks/scipy_comparison.pythroughput sweeps using the trough-at-threshold diagnostic and the clean-entry criterion documented indocs/SIMD_OPTIMIZATION_REFERENCE.md.Changes
kAvx512 (Zen4 / AMD Ryzen 7 7445HS)
kNeon (M1 / Apple Silicon)
kAvx2 (Kaby Lake / Intel i7-7820HQ) — this machine
kAvx (Sandy Bridge/Ivy Bridge — derived from kAvx2)
Derived from kAvx2: ÷2 for transcendental-heavy ops (exp-based); ÷1.5 for polynomial-only ops (Gaussian/Exponential/Laplace LogPDF) where FMA absence causes a smaller per-element cost delta.
kNone (no SIMD — comment-only)
Corrects incorrect 'L1d cache boundary' claim for T2=8192 (actual L1d boundary is N≈2048; 8192 is L2-resident). Notes T3=16384 sits at the L2→L3 boundary — the same failure mode fixed by NEVER in kAvx2/kAvx512 Uniform.
New documentation
docs/SIMD_OPTIMIZATION_REFERENCE.md: trough-at-threshold diagnostic, dispatch vs cache boundary distinction, L1 cache boundary on NEON/M1, clean-entry criterion for iterative calibration, NEVER threshold criteriadocs/SIMD_BENCHMARK_RESULTS.md: v2.0.2 cross-machine results, peak throughput table, speedup ratios vs scipy, accuracy table, benchmark commandsOpen items (tracked in #50, require Zen4 machine)
strategy_profile --largeon thermally-stable machineValidation
Closes #50 (partially — Zen4 Gaussian/Exponential items remain; see issue for validation steps)
Co-Authored-By: Oz oz-agent@warp.dev