Skip to content

Mallard 0.3.0

Choose a tag to compare

@MatthewBonanni MatthewBonanni released this 03 Oct 00:50
· 576 commits to main since this release

Mallard 0.3.0 adds three-dimensional simulation, periodic boundaries and a distributed-memory (MPI) solver that scales across GPU nodes.

3D

  • -DMallard_DIM=3 builds a 3D solver for tetrahedra, hexahedra, prisms and pyramids in any mix (#35–#37, #39). 2D and 3D builds live side by side.
  • Every numerical method carries over: Riemann solvers, MUSCL, TENO-E up to sixth order (#40), boundaries, viscous fluxes, Gmsh 3D meshes, VTU and restart output.
  • New examples: spherical explosion (#45) and the Taylor–Green vortex at Re = 1600 against the 512³ spectral DNS (#60), with an [integrals] monitor for kinetic energy, enstrophy and dilatation that uses the TENO reconstruction gradients (#72).

Periodic boundaries

  • [mesh] periodic = ["x", "y", "z"] on generated box meshes (all cell types), in 2D and 3D, for every scheme including TENO stencils that wrap across the seam, serial and distributed (#70, #75).

Distributed memory (MPI)

  • -DMallard_ENABLE_MPI=ON, then mpirun -n N Mallard -i input.toml (#33, #38, #41).
  • No rank holds the global mesh: a parallel HDF5 mesh format with the mallard-mesh-convert converter, distributed face matching, partitioning, cell migration and halo growth (#53, #54, #62, #66). Setup memory per rank now falls as 1/ranks; 64M quads set up on 16 ranks at 3.3 GB per rank.
  • Partitioning by Hilbert curve or by the dKaMinPar graph partitioner (-DMallard_ENABLE_KAMINPAR=ON, #46); GPU-aware halo exchange (-DMallard_GPU_AWARE_MPI=ON).
  • Output from every rank (.pvtu), and restart files that do not depend on the rank count (#41).
  • Results are bitwise identical on any number of ranks and threads (#49, #63).
  • Multi-GPU performance: TENO's shock-cell pass split into finer parallel work and the halo exchange overlapped with computation (#57, #65, #67). 2D TENO5 on A100s with GPU-aware MPI over InfiniBand: 4M cells run 1.7× faster on 16 GPUs than on 8, and weak scaling to 16 GPUs is 93% efficient.

Numerics

  • Low-Mach correction of the convective flux (Thornber et al. 2008), on by default with a cutoff Mach number [numerics] low_mach_cutoff = 0.1 (#78). It brings the Taylor–Green vortex kinetic energy onto the DNS and leaves the shock validation cases and the cylinder (St, Cd) unchanged.
  • Second-order viscous fluxes on triangles next to transmissive boundaries; quadratic vertex gradients (#55).
  • Deterministic flux accumulation without atomics (#49).

Usability

  • Clear console output: a build and setup summary, an aligned progress table (time step, throughput, ETA, minimum density and pressure, maximum Mach number, TENO troubled cells) and an end-of-run summary; input errors are one line naming the key and value (#88).
  • -DMallard_WARNINGS_AS_ERRORS=ON builds Mallard's own code with a strict warning set (#83).

Documentation

  • References for every method and validation dataset, with DOIs (#73, #90), and a CITATION.cff. This release is archived on Zenodo.
  • Design notes for MPI and for finite-rate chemistry (#32, #52).

Known limitations

  • 3D TENO uses about 50–80 KB of GPU memory per cell, so one A100 holds roughly 1M 3D cells at fifth order (#58).
  • The Mach 3 sphere develops a carbuncle on the axis where two symmetry planes meet with RHLL and TENO5 (#80); HLL is unaffected.
  • Gmsh files are read whole by every rank; convert large meshes to HDF5 for distributed runs.
  • The TENO setup cache works for serial runs only.

Full list of changes: v0.2.0...v0.3.0