Skip to content

v0.6.0

Latest
Latest

Choose a tag to compare

View all tags
@lsawade lsawade released this 18 Aug 13:34
v0.6.0
f9491ae
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

SPECFEM++ 0.6.0

🚀 TL;DR

New Features

  • Distributed-memory (MPI) parallelism — mesh decomposition, halo-exchange
    communication buffers, and an MPI-aware solver. Forward simulations now run across
    multiple ranks, validated against the homogeneous-elastic nightly benchmark.
  • Viscoelastic attenuation (2D/3D) — a standard-linear-solid (SLS) attenuation model
    with on-the-fly computation of relaxation times, plus a cookbook comparing attenuation
    on/off and the effect of the reference frequency.
  • 3D acoustic & acoustic–elastic coupling - conforming
    fluid–solid coupling, Stacey and acoustic-free-surface boundary conditions, SIMD
    execution.
  • 3D kernels - both acoustic and elastic kernels can be computed with and
    without attenuation.
  • New source formats + UTM mesh support — read SPECFEM3D-style FORCE and
    CMTSOLUTION files, optional source date-times, and UTM coordinate projection
    (including topography).
  • SAC seismogram output — alongside a reorganized io_backends layer and parallel
    HDF5 writing for VTK wavefields.

Usability & robustness

  • Simulation stability / CFL checks and mesh-resolution checks that warn before a
    run goes unstable or under-resolved.
  • Version header printed to the log so every run records the exact build.
  • Error handling improved with more informative error messages and a centralized error reporting system.

Under the hood

  • Completed the include/ → core/ migration: all library code now lives under
    core/specfem/.
  • New tag_dispatch / typed-storage machinery replacing the older FOR_EACH macros.
  • CLI11 replaces Boost::program_options; Kokkos updated to 5.1.1 and the
    in-house Kokkos abstraction wrappers were removed.
  • Performance: SIMD over DOF chunks, a software memory prefetcher, compile-time grid
    extents for chunked iterators, and assorted build-time reductions.

📋 What's New

Distributed-memory parallelism (MPI)

SPECFEM++ now runs forward simulations across multiple MPI ranks. The mesh is split with
a new in-repo xdecompose_mesh tool (METIS-backed), each rank reads its own partition of
the database, and ranks exchange shared-degree-of-freedom contributions through dedicated
communication buffers built from the mesh adjacency graph. The assembly, mass-matrix
computation, source/receiver location, and seismogram writing are all MPI-aware, and the
nightly homogeneous-elastic benchmark now exercises the MPI path end-to-end.

Below, on the left, is the MPI partitioning for the Mount St. Helens 3D
benchmark; on the right is a simulation of the same benchmark running across 4
MPI ranks.

partitioning

mt_st_helens_mpi.mp4

Viscoelastic attenuation

We added a standard-linear-solid (SLS) attenuation model for 2D and 3D media. Relaxation times
(tau_sigma, tau_epsilon) are fit on the fly — tau_epsilon via a Nelder–Mead
optimization against the target quality factor over the simulation frequency band — and
the attenuation contribution is folded into the stiffness computation through a Taylor
expansion stored in the assembly. Runtime configuration (number of standard linear solids,
reference frequency, frequency band) is read from the parameter file, and the code can emit
the realized Q(f) curve for inspection.

Two cookbooks demonstrate the feature: one comparing a simulation with attenuation on vs.
off, and one illustrating the effect of the reference frequency.

attenuation on/off attenuation by reference frequency

3D capabilities

3D continues to mature toward feature parity with 2D:

  • Acoustic media & fluid–solid coupling — conforming acoustic–elastic
    coupling, with displacement integration tests.
  • Boundary conditions — Stacey absorbing and acoustic-free-surface boundaries in 3D.
  • SIMD — vectorized execution enabled for 3D simulations.
  • Adjoint / sensitivity kernels — We now support 3D adjoint source computation and Frechet derivatives for both acoustic and elastic media, with and without attenuation.

Below we show the 3D elastic Kernel benchmark against Fortran. The top row shows
specfem++ kernels, the center row shows the reference Fortran kernels and the
bottom row shows the difference between the two. As expected the difference is
localized to the source and receiver regions.

attenuation on/off

Sources, receivers & I/O

  • FORCE and CMTSOLUTION readers — source definitions in the SPECFEM3D file formats,
    with read_sources refactored so new formats are easy to add.
  • UTM projection & date-times — sources and receivers can be specified in UTM
    coordinates (with topography), and sources can carry absolute date-times.
  • SAC seismograms — seismogram output in SAC format, on top of a reorganized
    io_backends layer (ASCII / HDF5 / ADIOS2 / NPY / NPZ / SAC).
  • Parallel HDF5 for the VTKHDF wavefield writer.

Robustness & developer experience

  • Stability and mesh-resolution checks warn when a simulation is likely to go unstable
    (CFL) or is under-resolved for the source frequency content.
  • The executable now prints a version header, so logs are self-documenting.
  • A large code-organization pass: the include/ tree was fully migrated into
    core/specfem/, the older FOR_EACH macros were replaced by tag_dispatch and typed
    storage, anonymous namespaces were removed (unity-build safe), Boost::program_options
    was replaced by CLI11, and Kokkos was bumped to 5.1.1.

🧪 Performance Benchmarks

We benchmarked performance for both the 2D and the 3D part of the code. 2D is
now 2x faster than the original Fortran code, and 3D in its first iteration of
performance optimizations reached 0.6 times the performance of the original
Fortran code. The full benchmark results are shown in the figure below.

attenuation on/off

📊 Statistics

  • 249 pull requests merged since v0.5.0 (Jan 16 – Aug 17, 2026)
  • 1546 commits since v0.5.0
  • 5 contributors

attenuation on/off

New Contributors

No new contributors this cycle — all returning.

What's Changed

👨‍🔬 New physics

  • Issue 1592 - Implements computation of tau_sigma for attenuation by @lsawade in #1608
  • Issue 1578 [Part 1] - Adds CR for acoustic media by @Rohit-Kakodkar in #1607
  • Issue 1591 - Maxwell Solid, Nelder-Mead-Algorithm, tau_epsilon computation by @lsawade in #1615
  • Issue 1595 - Implement Modulus and scale_factor computation by @lsawade in #1636
  • Issue 1581 - Enable 3D compute coupling kernel by @Rohit-Kakodkar in #1726
  • Issue 1594 part 3 - Implements Assembly::Attenuation and Assembly::FieldDerivativeStorage by @lsawade in #1765
  • Issue 1123 - 3D acoustic kernel by @icui in #1927
  • Issue-1903 Adds boundary conditions for 3D by @icui in #1914

🚀 Performance enhancement

✚ Enhancement

📚 Documentation

Refactor

  • 1554 - Move algorithms to core/ by @icui in #1562
  • 1554 - Move boundary to core by @icui in #1566
  • 1554 - Move datatype to core (small refactor) by @icui in #1567
  • 1554 - Move execution to core by @icui in #1568
  • 1554 - Move parallel configuration to core by @icui in #1569
  • 1554 - Move quadrature to core/ by @icui in #1570
  • 1554 - Move utilities to core/ by @icui in #1571
  • 1554 - Move parameter_parser to core (small refactor) by @icui in #1572
  • 1554 - Move kokkos_kernels to core/ by @icui in #1575
  • Issue 1554 - Move most code from include/ to core/ (excluding io, medium and enumerations) by @icui in #1584
  • Issue 1330 - Removes Kokkos wrappers from core - Part 1 by @Rohit-Kakodkar in #1590
  • Issue 1563 - Remove anonymous namespaces, Part 1 by @lsawade in #1605
  • 1554 - Rename structs in kokkos_kernels by @icui in #1588
  • 1554 - Move medium to core Part II: refactor medium_container by @icui in #1612
  • 1554 - Move io to core by @icui in #1617
  • 1554 - Move enumerations to core Part I by @icui in #1631
  • Promote RegisterArray to datatype namespace by @Rohit-Kakodkar with @Copilot in #1625
  • Issue 1330 - Deprecate Kokkos abstractions by @Rohit-Kakodkar in #1632
  • 1554 - Move enumerations to core Part II by @icui in #1635
  • 1064 - Update specfem/compute with new tags syntax by @icui in #1647
  • CLI11 replacing Boost::program_options. by @lsawade in #1644
  • Fix tpp imports. by @icui in #1648
  • Issue 1064 - Update tags for time_marching. by @icui in #1649
  • Issue 1626 - Moves setup and constants to core, deprecates globals, default_file by @lsawade in #1650
  • Refactor sources to disentangle some things by @lsawade in #1683
  • Moves DomainView to datatype namespace by @Rohit-Kakodkar in #1679
  • Reduced compiled time by 15% by just not doing a superheader for asse… by @lsawade in #1684
  • Merge Edge and Face types within assembly by @Rohit-Kakodkar in #1691
  • 1064 tags - Template point and compute with specfem::tags::Tags by @icui in #1717
  • 759 - Move io libraries from io to io_backends by @icui in #1814
  • Issue 1581 - Updates database processor names by @Rohit-Kakodkar in #1818
  • 1719 - Introduce specfem::tags::expand to simplify creation by @icui in #1807
  • Issue 1362 - Refactor read_sources for simpler addition of new formats. by @lsawade in #1858
  • ...ChunkFaceViewType types created + ...ChunkElementViewType refactor by @int-ptr-ptr in #1876
  • Added LayoutType to chunk_ndim_view by @int-ptr-ptr in #1886
  • Issue 1813 - Refactor compute stiffness kernel by @Rohit-Kakodkar in #1878
  • Issue 1867 - Refactor coordinates to allow for better conversion by @lsawade in #1873
  • Minor organization to scripts, + pytest testing for scripts by @int-ptr-ptr in #1912
  • 1945 - Template specfem::point::kernels with Tags by @icui in #1968
  • Issue 1954 - Disentangling unit-tests and integration-tests by @lsawade in #1974
  • Restructure the undo-attenuation adjoint solver for readability by @Rohit-Kakodkar in #2015

🐛 Bug Fixes

Configuration/CMake

🔧 Maintenance

  • Fix: Clang Memory Limit by splitting updated wavefields and instantiation by @lsawade in #1745
  • 1702 - Add SPECFEM_MPI_SAFE_CALL by @icui in #1761
  • 1819 - Avoid unnecessary print() calls when not printed. by @icui in #1834
  • 1819 - Update collective output by @icui in #1843
  • Issue 1771 - Fix units to use macros for redundant code bits. by @lsawade in #1856
  • Issue 1330 - Update Kokkos to 5.1.1 by @Rohit-Kakodkar in #1860
  • Remove unused point::stress_integrand by @icui in #1969
  • Cleans up log output of Receivers/Sources/Source Time functions and adds LocationResult debug printing by @lsawade in #1964
  • Updates to Kokkos release 5.2.0 by @Rohit-Kakodkar in #2018
  • Issue 1997 - Includes VTK & optional SIMD build in the Dockerfile by @lsawade in #2030
  • Update project version to 0.6.0 by @lsawade in #2032
  • Release Candidate v0.6.0 by @Rohit-Kakodkar in #2033

🧾 New tests

Full Changelog: v0.5.0...v0.6.0

Contributors

lsawade, Rohit-Kakodkar, and 3 other contributors
Assets 2
Loading