Skip to content

ace-jax 0.5.0

Choose a tag to compare

@jameskermode jameskermode released this 27 Oct 12:06
· 53 commits to main since this release
8ab3cca

Highlights. Linear ACE runs on CUDA GPUs with a CUDA kernel: in LAMMPS,
in the ASE calculator and in batches. In LAMMPS on an A100 it is 2.9 to 8.0
times faster than the compiled JAX program, and linear ACE o4d20 costs 1.23
µs/atom-step. The C++ kernel is now the default for CPU LAMMPS bundles, and
it also runs in the ASE calculator and in JAX-MD. BatchCalculator evaluates
many small structures in a few compiled calls, and batched MD runs with
JAX-MD. Learned radials are approximately 3 times faster.

Upgrading from 0.4.1. Saved models are unchanged.

  • A CPU bundle of a linear ACE model now uses the C++ kernel by default.
    LAMMPS then needs LAMMPS_JAX_FFI_HANDLERS, which aj export prints.
    Export with --no-ffi (ffi=False) for the previous bundle, which needs no
    variable.
  • aj fit --learn-radial runs 300 steps by default (was 40), with a new gate
    score, so the same command gives a different fit. Set --radial-steps 40
    for the previous number of steps.
  • The fast-neighbours extra is now native. The old name works until the
    next release but one.
  • For an NVIDIA driver 580 or later, the cuda13 extra installs nvcc for
    the CUDA kernel. With the cuda extra, load a CUDA toolkit or module.

Changed

  • Learned radials: longer runs and a force-aware gate. --radial-steps
    now defaults to 300 (was 40), and the new --radial-reprofile-every
    (default 100; was fixed at 10) sets how often the hyperparameters are fitted
    again. The gate score now weights energies and forces as the learning
    objective does, so the force errors count. The previous score was a mean
    for each quantity, so the energies decided it. The fit log and
    radial_info.json give the polynomial span that is used.
  • Learned radials are approximately 3 times faster. The residual pass uses the
    model's own energies and forces at the coefficients, not the full design
    rows. The hyperparameters are fitted again by L-BFGS, not by 300 Adam steps.
    The L-BFGS fit is also better converged.
  • A cuda13 extra. For an NVIDIA driver 580 or later, it installs
    jax[cuda13] and NVIDIA's nvcc wheel, which compiles the CUDA kernel. The
    cuda extra is unchanged (jax[cuda12]): the kernel then uses the nvcc of
    a CUDA toolkit or module.
  • The radial table is sized to an accuracy. radial_table=True (and the
    C++ and CUDA kernels, which turn it on) now uses 4000 intervals or more: enough for
    an error of 1e-9 in the radial values. Most models keep 4000 intervals. A
    species-embedded SiGe model gets 11585, and its force error goes from
    2.7e-7 to 1.4e-8 of the largest force, at the same speed. The search stops
    when more intervals give no smaller error in the derivative of the table:
    past that point, float64 noise in the sampled radial grows with the number
    of intervals. A float sets another tolerance; an int still sets the number
    of intervals.
  • fast-neighbours is now native. The native extra installs the same
    packages; fast-neighbours remains as an alias until the next release but one.
  • The C++ kernel is the default for CPU bundles. export_lammps and
    aj export take ffi="auto": a CPU bundle of a linear ACE model (categorical
    or species embedding) uses the kernel when it is float64, lean, has the
    matrix layout, lammps-jax is 0.2.1 or later and a C++17 compiler compiles
    the kernel. Otherwise the bundle uses the XLA program, aj export prints the
    reason, and the bundle records it as ace_jax.ffi_reason. A missing compiler
    no longer stops the export. --ffi (ffi=True) requires the kernel and
    --no-ffi (ffi=False) turns it off. GPU bundles are unchanged.
  • Faster GPU compiles for single-species ACE models. The analytic radial
    is now a matrix multiply when the model has one species. For batched MD on
    an A100, the compile now takes approximately 5 seconds, not 10 minutes.
    Multi-species models are unchanged.

Added

  • A CUDA kernel for linear ACE. On a CUDA GPU, ffi=True runs a CUDA
    kernel in float64 or float32: in ACECalculator, BatchCalculator, and
    LAMMPS bundles (export_lammps(ffi=True), aj export --ffi). It supports
    categorical models with any number of species and species-embedded models.
    On an RTX 4000 Ada with 4096 Si atoms, it is 3.4 to 4.8 times faster than
    the XLA program in float64, and 2.7 to 3.4 times faster in float32. In
    LAMMPS on an A100 with 32,768 Si atoms, it is 2.9 to 8.0 times faster in
    float64 (o3d8 to o4d20) and 2.3 to 5.3 times in float32; linear ACE o4d20
    costs 1.23 µs/atom-step, against 2.42 with patched ML-PACE. nvcc compiles
    it the first time; aj native --build compiles it before use.

  • Batched evaluation. ace_jax.batch.BatchCalculator(model).calculate(structures)
    gives the energy, forces and stress of many structures in a small number of
    compiled calls: structures of similar size are padded into one batch, and
    the neighbour lists of a batch are made in one step. It accepts the
    evaluation options of ACECalculator and agrees with it to roundoff. On a
    GPU, 1024 Si cells of 2 or 8 atoms are 35 to 72 times faster than
    ACECalculator one structure per call, 64-atom cells 6 to 11 times. On a
    CPU it needs the C++ kernel (ffi=True): 25 to 35 times for 2 and 8
    atoms, 3.5 times for 64 (with matscipy-neighbours' batched build; 19 to 52
    and 2.4 times without it). aj eval uses it when no posterior or GP model is
    given. Guide: "Evaluate many structures".

  • Batched MD and relaxation with JAX-MD. JAX-MD's integrators and
    optimisers run over a batch of structures with jax.vmap:
    BatchCalculator.neighbour_lists(structures, skin) gives the padded batch
    and its neighbour lists (any cell size, rebuilt by update when an atom
    moves skin / 2), BatchCalculator.energy_fn() the energy of one structure,
    and ace_jax.batch.initial_momenta, zero_padding and temperature keep
    the padding atoms out of the dynamics and the temperatures. With ffi=True
    the C++ kernel runs under jax.vmap (the batch becomes one threaded kernel
    call): on 32 cores, MD of 64 Si cells of 8 or 64 atoms is 37 to 48 times
    faster than with the JAX model. BatchNeighbours.update_fn() rebuilds the
    lists inside the compiled loop (one batched build, under lax.cond).
    Give initial_momenta the positions' dtype for a float32 batch.
    On an A100, batches of 64 to 216 atom cells are 18 to 21 and 8.5 to 9 times
    faster than ACECalculator; a GPU call holds up to 262144 atoms
    (max_batch_atoms), approximately 1 GB in float64 for the Si model.

  • The C++ kernel in the JAX-MD adapter. jaxmd.potential(..., ffi=True)
    runs the kernel on a CPU: 10 to 18 times faster MD than the JAX model for
    216 to 1728 Si atoms on 32 cores. The adapter and the batched path now share
    one energy core, so the batched path also compacts the neighbour slots
    (BatchNeighbours.energy_fn), evaluates rows in blocks, and gives NaN, not a
    truncated list, when a structure outgrows its neighbour lists.

  • The C++ kernel in the ASE calculator. ACECalculator(..., ffi=True)
    calculates energy, forces and stress of a linear ACE model with the C++
    kernel on CPUs, on all cores (XLA's thread pool; ffi_threads sets fewer).
    On 32 cores it is approximately 17 times faster than the JAX model at 1,700
    to 4,100 atoms (linear ACE o4d20). It uses the calculator's neighbour lists
    and skin, and the radial table. ffi="auto" uses it when possible and gives
    the reason in calc.ffi_reason when not. The default stays the JAX model,
    which also gives derivatives with respect to the coefficients and second
    derivatives; the kernel gives position derivatives only.

  • aj export --print-env BUNDLE for kernel bundles. It prints the
    LAMMPS_JAX_FFI_HANDLERS line on the machine that runs LAMMPS. If the kernel
    library is not on that machine, the command compiles it there.

  • The native extra and aj native. pip install "ace-jax[native]"
    installs the C++ neighbour list (matscipy-neighbours). Its compiler also
    builds the C++ kernel. aj native shows the compiler, flags, CPU, kernel
    cache and neighbour list; aj native --build compiles the kernel now (on a
    cluster: on a compute node).

Fixed

  • ACECalculator no longer fails on a structure with no cell (a cluster with
    a zero cell): ASE's get_volume raised. The stress needs a non-zero volume.
  • Skin-list rebuilds on the CPU were slow. Matching each edge to its reverse
    (two sorts over every skin edge) took approximately 90% of a rebuild: 93 of
    106 ms at 4,096 atoms. The match only helps on a GPU, so the CPU now uses the
    scatter force assembly, which is as fast. A rebuild of 4,096 atoms now takes
    14 ms with the native extra (was 102 ms) and 159 ms without it
    (was 249 ms).