Skip to content

DKX 2.9.0

Choose a tag to compare

@rogeriojorge rogeriojorge released this 07 Oct 13:56
· 53 commits to main since this release
8f0e121

Exact direct solves for Fokker–Planck decks that fit, a coupled preconditioner
that auto chooses when memory allows, native and validated Phi1 with an
adjoint, spectral angles for monoenergetic databases, a better default CPU
thread split, and case inputs without a schema key.

  • Default CPU threads: with neither DKX_CORES nor NPROC set, XLA's pool gets min(8, cores) and BLAS min(4, cores) threads, counting the cores the process may run on (taskset aware, not os.cpu_count()). Before, BLAS was single-threaded, which left XLA's sequential LAPACK custom calls (getrf, trsm) on one core. Explicit DKX_CORES, NPROC and BLAS variables are unchanged, and forced host devices keep one BLAS thread. Office Xeon, warm, main to this change: W7-X monoenergetic 2.55 to 1.67 s pinned to 4 cores and 2.60 to 1.75 s unpinned; NCSX full FP 5.2 to 5.0 s and 5.3 to 4.8 s; sparse direct unchanged at 23.4 s; the bounce-averaged surrogate unchanged. A 14-core laptop is within noise.

  • tools/benchmarks/cross_code_speed.py --dkx-xla-threads N sets DKX's XLA pool apart from YANCC's. On four pinned x86 cores, XLA's pool and OpenBLAS compete inside every LAPACK call, and one XLA thread makes DKX's warm monoenergetic solve 2–3× faster (W7-X 3.4 to 1.6 s, HSX 19.5 to 6.8 s; MONKES 0.32 s and 1.5 s).

  • monoenergetic_database now uses spectral (Fourier) angular derivatives when the deck does not set thetaDerivativeScheme/zetaDerivativeScheme. A deck that sets them keeps its choice. On the matched W7-X/HSX benchmark this is the discretization MONKES uses, and it converges like MONKES: HSX reaches 3% at 19x41x64 instead of 25x51x96, and W7-X is at 4e-5 one rung above 15x31x48 (finite differences: 0.6% at 25x51x96). Values at a fixed grid change within discretization error. To pair a database with a full-kinetic solve of the same deck, set the schemes in the deck.

  • The full-band structured direct route (build_tier1_solver) builds its Legendre rows one at a time inside the elimination scan, stores only the Schur LUs, and regenerates the off-diagonal blocks in the substitution sweeps. Answers are unchanged (1e-12). On W7-X 15x31x48 the peak RSS falls from 1.95 GB to 0.94 GB and the first solve without a compilation cache from 8.2 s to 3.7 s; the warm solve is 8% slower. With the default persistent cache the first solve takes 1.4 s.

  • tools/benchmarks/cross_code_speed.py has a cached stage that times DKX's first solve with the persistent compilation cache on.

  • Phi1 is promoted from compatibility_only to validated_limited. On the geometryScheme 4 Phi1 example deck and a trace-carbon variant, DKX matches Fortran SFINCS v3 to 2.0e-6 over an eight-rung resolution ladder (validation/phi1_sfincs_benchmark_v1.json). stable_candidate is withheld: there is one geometry, Fokker-Planck collisions only, zero Er, and the trace-impurity flux is resolved only to about 3% in Nx.

  • dkx.phi1.phi1_solution gives the converged Phi1 state with a matrix-free implicit adjoint: GMRES on the transposed Jacobian (a VJP of the coupled residual) on the active Legendre-truncated subspace, preconditioned by the last Newton step's transpose preconditioner. On a native phi1 = "kinetic" case the gradient matches central finite differences to 1.2e-9 and the Taylor remainder falls as h^2 (ratios 4.02, 4.01).

  • Case inputs no longer carry a schema key. Older files with schema = 1
    still load (the key is ignored); other values are refused. Case IDs are
    unchanged.

  • dkx schema is renamed dkx template (schema stays a hidden alias for one
    release). Case no longer has a schema attribute and SCHEMA_VERSION is
    no longer exported; case IDs are unchanged.

Solvers

  • Full Fokker–Planck and improved Sugama decks with DKES trajectories, no
    tangential drifts and no Phi1 now have an exact structured direct route: a
    block elimination along the Legendre index whose blocks couple all species
    and speeds (dkx.structured_direct.build_coupled_solver). It eliminates from
    the highest Legendre mode down, lifts the momentum null space of the L = 1
    block with a rank-Nspecies term removed again by Woodbury, and handles every
    constraintScheme border. method="auto" takes it within the structured
    memory budget and 150 GFlop per right-hand side; on the upstream suite it
    solves transportMatrix_geometryScheme2 in 2.8 s against 33 s (1,181 GCROT
    iterations) and transportMatrix_geometryScheme11 in 9.9 s against 48 s, and
    leaves geometryScheme4_2species_noEr and the W7-X/HSX decks, where Krylov is
    faster or the blocks do not fit, on recycled Krylov. A speed-triangular sweep
    was measured first and is not exact in this speed basis (lower triangle 1-3%
    of the diagonal, electron-ion block as large as its diagonal).
  • preconditioner="coupled": the speed-coupled elimination as a Krylov
    preconditioner, exact for Legendre blocks L < 2 and (species, x)-diagonal
    (float32) in the tail; E_r and drift L ± 2 terms are dropped from the
    factored operator. Iterations fall from 46 to 17 on a 396k-unknown HSX FP
    deck at equal wall time.
    It keeps the L-diagonal magnetic drifts at L <= 2 (23 to 17 iterations on
    the reduced W7-X drift deck) and stores tail bands as per-species blocks times
    the speed scale, Nx times smaller.
    The tail is now eliminated and swept with lax.scan (padded with identity
    blocks where Nxi_for_x truncates), its bands are kept as two scalars per L
    over shared float32 streaming and mirror blocks, and the dense L < 2 Schur
    step is formed one column pair at a time: the compiled HSX FP production
    factorization drops from 11.5 GB output + 26.6 GB temporaries to
    11.7 GB + 4.4 GB.
    Its tail and dense L < 2 blocks are now factored in float32: on the
    production HSX FP deck (1.88M unknowns, 8 cores) 17 iterations, 508 s and
    16.6 GB, against 1,100 s and 15.0 GB for the coarse preconditioner.
  • method="auto" now preconditions Fokker–Planck and Sugama Krylov solves with
    coupled when its estimated peak fits the Krylov memory budget (default: half
    the available memory), else coarse.
  • dkx.solve keeps its API; the structured route's applicability tests and
    memory model moved to dkx.structured_direct. Structured solves refine
    against the pinned operator (identity rows on truncated Nxi_for_x DOFs).
  • Native [physics] phi1 = "kinetic" runs the coupled kinetic + quasineutrality + gauge Newton–Krylov solve on profile cases and is accepted by the nonlinear residual; dkx convert maps kinetic-only Phi1 decks to it. "full", adiabatic species, quasineutrality option 2 and the ambipolar workflow stay refused. Status remains compatibility_only.