Skip to content

SCPN Fusion Core v3.9.0 — QLKNN-10D Real-Data Training Pipeline

Choose a tag to compare

View all tags
@anulum anulum released this 21 Feb 21:58
· 1218 commits to main since this release
v3.9.0
b632084

Highlights

This release introduces the complete pipeline for retraining all ML surrogates on real gyrokinetic data (QLKNN-10D, 300M QuaLiKiz flux calculations), replacing the synthetic training data that produced misleading metrics.

Added — QLKNN-10D Real-Data Training Pipeline

  • Data Acquisition (tools/download_qlknn10d.py): Downloads QLKNN-10D from Zenodo (DOI: 10.5281/zenodo.3497066) with SHA256 verification and resume support
  • Data Pipeline (tools/qlknn10d_to_npz.py): HDF5 → training .npz with column mapping, gyro-Bohm → physical conversion, regime classification (ITG/TEM/stable), stratified 90/5/5 splits
  • MLP Training (tools/train_neural_transport_qlknn.py): JAX trainer with verification gates — refuses to save weights if test_relative_l2 >= 0.05
  • FNO Spatial Data (tools/generate_fno_qlknn_spatial.py): Trained MLP as oracle for (equilibrium, transport_field) pairs
  • FNO Training (tools/train_fno_qlknn_spatial.py): JAX FNO targeting relative L2 < 0.10 (was 0.79 on synthetic data)
  • GPU Diagnostic (tools/check_gpu.py): JAX GPU, PyTorch CUDA, and Rust wgpu availability
  • Validation Suite (validation/validate_transport_qlknn.py): Per-output and per-regime metrics vs published benchmarks

Added — Architecture Improvements

  • Variable-Depth MLP Loader: neural_transport.py auto-detects depth from .npz keys, backward compatible
  • GELU Activation: Hidden layers use GELU (matching JAX/PyTorch training convention)
  • GPU Optional Dependency: pip install scpn-fusion[gpu] installs jax[cuda12]

Fixed

  • Removed inflated FNO claims ("0.9997 TGLF correlation", "98% suppression efficiency") from all documentation
  • Fixed corrupted UTF-16 entries in .gitignore
  • validation/collect_results.py reads actual metrics instead of hardcoding values
  • Hardened sync_metadata.py to catch all version reference files

Competitive Roadmap

All 8 competitive gap items (G1–G8) now addressed:

Gap Status
G1: GPU Multigrid ✅ fusion-gpu crate + gs_solver.wgsl
G2: JAX Autodiff Transport ✅ jax_transport_solver.py + fusion_nmpc_jax.py
G3: QLKNN-10D Weights ✅ Full pipeline (this release)
G4: FNO on Real Data ✅ Full pipeline (this release)
G5: Gymnasium Wrapper ✅ gym_tokamak_env.py, Tokamak-v0
G6: EPED Pedestal ✅ eped_pedestal.py with KBM constraint
G7: Fokker-Planck RE ✅ fokker_planck_re.py, 1D momentum-space
G8: Multi-Fragment SPI ✅ spi_ablation.py, Lagrangian + Parks ablation

Full Changelog: v3.5.0...v3.9.0

Assets 2
Loading