Skip to content

peclet 1.5.0 — a faster distributed VoF step, solids in scalar transport, and balanced partitions by default

Latest

Choose a tag to compare

@eajfpeters eajfpeters released this 23 Oct 01:52
· 22 commits to main since this release

peclet-flow 1.4.0 · peclet-coupling 1.2.0 · metapackages peclet and peclet-cu13 1.5.0.
Unchanged and still pinned: peclet-dem 1.1.0, peclet-pnm 1.0.3, peclet-voro 1.0.4,
peclet-morton 1.0.2, peclet-geom 1.0.1, peclet-amr 0.2.0 (their commits since their tags touch
docs, scripts or packaging metadata only). They keep their PECLET_CORE_TAG v1.3.0 pins: the pin is
build-time only, and the one run-time contract between packages that core touched since — the
weighted ORB that flow and dem build the shared CFD-DEM partition from — is byte-identical from core
v1.3.0 to v1.7.0 (SplitAxis::Balanced applies only to unweighted decompositions that ask for it).
peclet-halo / peclet-core 1.5.0, 1.6.0 and 1.7.0 were released on their own (sections below);
flow and coupling pin v1.7.0, and peclet[mpi] now installs peclet-core 1.7.0.

peclet-flow 1.4.0

Added

  • Cut-cell scalar transport with solids (opt-in: add_scalar(name, D, cutcell=True)). A
    transported scalar sees the immersed bodies as cut cells with a probe-flux wall condition, second
    order for Neumann, Dirichlet and Robin walls (set_scalar_wall), conjugate transport into the
    solid (set_scalar_solid, get_scalar_solid), sources (set_scalar_source), a mean-gradient
    closure for periodic beds (set_scalar_mean_gradient, diagnostics.scalar_mean_flux), and a
    steady solve (solve_scalar_steady) beside the transient advance; set_scalar_residual_tolerance
    (the velocity solver's criterion, one spelling), diagnostics.set_scalar_max_iterations(max_iter);
    per-body wall fluxes diagnostics.scalar_wall_flux, budgets and
    solver statistics diagnostics.scalar_budget / scalar_census, the cut-cell record
    diagnostics.scalar_geometry / scalar_facets. Conservation is exact in the discretization; the
    solver residual is reported, never fixed up. Staggered grids and the collocated cut-cell
    projection; refused by name: the collocated 'ghost' scheme, collocated open faces, porous
    continuity, moving scenes. Legacy scalars (cutcell=False, the default) are unchanged.
  • diagnostics.set_pressure_vcycle_precision('fp64' | 'fp32' | 'auto') and
    diagnostics.pressure_vcycle_precision — an FP32 V-cycle below the exact FP64 PCG/FCG, with an
    FP64 restart (latched for the run) on a cap or breakdown; single rank and distributed.
  • diagnostics.set_pressure_initial_guess('zero' | 'projected') (opt-in; default 'zero'):
    A-norm projection onto the last one or two pressure solutions.
  • diagnostics.set_pressure_prefactor_hoist(bool), diagnostics.num_pressure_prefactors,
    diagnostics.num_pressure_chebyshev_restarts (developer tier).
  • With np > 1, init_mpi prints one line on rank 0 naming ranks whose OpenMP threads straddle L3
    (CCD) domains (no result effect). initMpi(dec) refuses, on every rank, a decomposition the ranks
    do not share.

Changed (named numerics changes)

  • The GPU pressure V-cycle defaults to 'auto': FP32 where eligible on device backends, at
    every np, with the FP64 restart; host 'auto' is 'fp64', so host results are bitwise unchanged.
    Device fields move in the last digits (~1e-13 relative); 66 probe configurations 0 violations,
    2000 steps 0 restarts. ~5 % per step on one H100 (16.9 → 16.1 ms), −28 % projection on an
    RTX 5080, never slower at np > 1. 'fp64' restores the previous solve.
  • The default decomposition (set_decomposition(0)) is the AUTO policy: the legacy aligned ORB
    where it is exactly balanced (bit-identical there), otherwise a balance-aware coarse-first
    partition (core SplitAxis::Balanced), otherwise the better of legacy and the balance-aware
    aligned ORB; it never cuts an axis the legacy ORB leaves whole. np > 1 partitions change where
    the legacy split is inexact
    (bubble column np 3/12/24): fields within 5.9e-14 of the legacy
    partition, iterations identical; the bubble column at 24×1 was 2:1 imbalanced under legacy.
    set_decomposition(1) is the legacy ORB; explicit levels ≥ 2 are unchanged (2394/2394 published
    partitions byte-identical). The policy keeps legacy's set of split axes unless legacy's max/mean
    load is ≥ 5/4; this guard reverts 14 census configurations to the legacy partition (128×96×64
    at np 12 among them). The box-shell launch of the auto 12×2 column no longer runs serially on one
    thread (6.2 ms/step on genoa); bitwise.
  • Host backends use the 'direct' pressure bottom where eligible (single rank: FP32
    block-tridiagonal factor + FP64 FCG, replacing GraphAMG; distributed: solved redundantly on every
    rank from the allgathered bottom). 50-step difference 3.3e-14 single rank, 2.3–3.6e-14 at np > 1,
    pressure iterations identical; set_pressure_bottom_solver('algebraic') restores GraphAMG.
    Workstation 8×3: 61.4 → 53.6 ms/step.
  • VoF tier-3 curvature fallback (core 1.5.0's moment kernels): a recorded round-off change, 50-step
    max relative 2.7e-14 host / 2.5e-14 CUDA, iterations identical; tier 3 3.55 → 0.58 ms on an
    RTX 5080.
  • Host pressure-MG sum reductions run in pencil order; the Chebyshev driver re-estimates its spectral
    bounds from the previous estimate after a variable-density rebuild (cold re-estimate on a guard,
    counted); VoF block statistics are reduced in one launch per step (device: fixed 512-cell tiles) —
    diagnostics order only.
  • Scalar and phase-change energy inputs are physical under an armed extent (a fix of the
    physical-units contract): add_scalar's diffusivity, mass fluxes, latent heat, conductivity,
    ρc_p and the closures on ρ/μ/forces are converted at the boundary; the getters report physical.
    Scripts that passed cell-unit values to a solver built with extent= change result.
  • VoF blocks under a physical domain (Solver(cells, extent=…) with enable_vof_blocks): the
    step handed its internal dt to the physical-unit block advector, which converted it a second
    time (the first step hit the Weymouth–Yue cap), and the block CSF kept a σ from before the first
    set_dt (1/t_ref² too strong). Both fixed; cell-unit runs are bitwise unchanged (ctest
    vof_blocks_physical_units).
  • Open boundaries keep their high-side face plane. Ghost fills of the MAC face fields wrapped
    every axis, so a non-periodic high-side face (outflow or inflow) carried the low boundary's
    face; the collocated projection and the collocated VoF bridge exposed it, and legacy scalars
    read the wrapped plane. Periodic axes are untouched (ctest open_face_plane, np 1–3).
  • Scalar inflow. A scalar's ghosts are filled before the old state is frozen, so the first
    step after set_field (or a coupling or VoF write) no longer upwinds from stale ghosts; at a
    Dirichlet face that is also a velocity inflow the advection upwinds the prescribed value, not the
    Dirichlet reflection (a plug's inventory was U t + 0.5 cells; now U t to 6e-14, ctest
    scalar_inflow). Results change for scalars written before the first step (state hash
    scalar). Trade: on a smooth inflow profile the first interior face carries an O(h) error
    (gh/6 coded limiter, gh/4 second-order upwind).

Changed (performance, bitwise)

  • Distributed host step: the pressure-MG stencil exchanges are faces-only with rank-local periodic
    axes read in-kernel; core 1.6.0's halo transport (persistent requests, one pack launch, on-node
    shared memory: 10.9–12.5 → 1.1–1.7 µs per exchange); fused host Krylov reductions; host VoF
    kernels and momentum ghost fills in fewer launches.
  • Device step: no global synchronization inside step() (ctest device_sync_budget), GPU-resident
    coarse tail with the bottom factor on a side execution instance.

Known limitations

  • Cut-cell scalars: particles in contact do not converge at R/h 8–32 (no contact model yet);
    collocated open faces are refused.
  • Distributed host runs: threads per rank must divide the cores of an L3 domain; 8×3-type layouts
    on 8-core CCDs are slow (+17.6 ms kernel time on genoa), whatever the partition.

peclet-coupling 1.2.0

Changed

  • The void-fraction floor CfdDem(eps_min=…) defaults to 0.05, down from 0.25 (also the default
    of _coupling.compute_void_fraction). The floor is a division guard for the 1/ε factors in the
    drag closures and the porous projection, not a physical packing limit: the minimum voidage
    depends on particle shape and size distribution (spheres ~0.36, wide bidisperse mixes and
    ellipsoids ~0.25, space-filling shapes → 0), so 0.25 capped the drag of dense non-spherical and
    polydisperse beds. Results change wherever a cell's ε fell below 0.25. For a noisy deposit at a
    cell of about one particle diameter, set smooth_length (~1.5 d_p); pass eps_min=0.25 to
    reproduce earlier runs.
  • Pins core v1.7.0.