Integrate CitcomS predictor-corrector into unified Eulerian SUPG by gthyagi · Pull Request #689 · underworldcode/underworld3

gthyagi · 2026-09-05T16:56:21Z

Summary

This is a focused follow-up to #673. It adds the Zhong/CitcomS P1
predictor-corrector as an optional mode of the existing Eulerian SUPG solver,
rather than maintaining a second SUPG implementation.

After this change there is one public class,
uw.systems.AdvDiffusionSUPG, and one implementation in
systems/advection_diffusion_eulerian.py. The former advdiff_supg module is
only a compatibility import. CN, backward Euler, BDF2/BDF3, and the CitcomS
predictor-corrector therefore share the same solver lifecycle, residual
assembly, PETSc configuration, snapshot registration, and documentation.

This PR deliberately targets feature/eulerian-supg-transport, not
development, so the predictor-corrector can be reviewed together with the
Eulerian transport implementation it extends.

Why

The Zhong et al. (2008) mantle-convection benchmark requires the CitcomS-style
temperature update, but adding it as an independent solver produced duplicated
SUPG assembly, state handling, caches, and documentation. It also made matched
CN/BDF/PC2 benchmark comparisons harder because nominally common behavior lived
in different classes.

The integration preserves the CitcomS numerical choices:

continuous P1 temperature on simplex cells;
directional steady SUPG stabilization;
positive lumped mass;
gamma=0.5;
two predictor-corrector corrections;
explicit advection/diffusion stability bound; and
caller-visible Tdot state for restart and benchmark output.

It does not replace the generic transient SUPG parameter or alter the existing
implicit CN/BDF equations.

Public numerical contract

Method	Selection	Timestep policy	Restart state
CitcomS predictor-corrector	`time_integrator="citcoms"`	`0.9 * min(dt_adv, dt_diff)`	`T`, `Tdot`, startup state
Converged predictor-corrector	`time_integrator="pc_converged"`	same stability estimate	`T`, `Tdot`, startup state and correction controls
Crank-Nicolson	`order=1`, `theta=0.5`	field-change accuracy estimate	`T`, history, timestep and estimator state
Backward Euler	`order=1`, `theta=1.0`	field-change accuracy estimate	`T`, history, timestep and estimator state
BDF2/BDF3	`order=2/3`	field-change accuracy estimate	multilevel history, timestep and estimator state

CitcomS mode does not allocate the unused implicit multistep history. Repeated
updates reuse the assembled simplex geometry and predictor-corrector work
vectors.

Bugs found during validation

Symptom	Root cause	Change
A coarse empty-rank test poisoned the following MPI mesh load	`DMPlexIsSimplex` was interpreted rank-locally, so populated and empty ranks built different coordinate finite elements and consumed different PETSc message tags	included from independently reviewed PR #691
Fresh-process CN/BDF restart failed before solving	lazy generic SUPG geometry (`_h_cell`) was not registered before snapshot metadata restoration	register generic geometry during solver construction before state restore
Some MPI assertions failed although the global solution agreed	tests compared rank-local pulse amplitudes and maxima	use global norms/maxima and collective finite/equality assertions
Generic SUPG differed between serial and MPI on the same coarse mesh (#687)	`mesh.cell_size()` used distance to the nearest centroid available on each rank	included from independently reviewed PR #692
Unsupported PC2 partitions could fail on one rank before peers reached a collective	simplex-layout rejection was rank-local	make partition validation collective and report rank/cell/dimension context

The complete commit stacks from #691 and #692 are included here as explicit
cherry-picks, while remaining available as independently reviewable generic-mesh
PRs. This makes #689 self-contained for the eight-rank empty-partition and
high-Peclet SUPG validation without importing unrelated development commits.
The own-cell calculation from #692 adapts only the mesh-size part of 68e545fd
from the Navier-Stokes SUPG work associated with #687.

Independent accuracy checks

The validation does not use a long coupled A1 run as its oracle. Prescribed
velocity and diffusion problems isolate transport on tiny triangle,
tetrahedron, and spherical-shell meshes.

Check	Result
Manufactured 2-D/3-D advection, diffusion, and combined pulse cases	passed analytical accuracy/refinement bounds
Quarter-turn Gaussian rotation	passed boundedness, integral, centroid phase, and L2 checks
Spherical radial diffusion, fine mesh	2.66% relative L2 error
PC2 update versus independently assembled closed recurrence	maximum field/rate discrepancy below `4e-14`
Actual UW3 Crank-Nicolson versus exact semidiscrete diffusion	temporal order 2.00 in triangles and tetrahedra
CN maximum discrepancy from its exact amplification map	`1.55e-14`

Important PC2 limitation

The fixed two-correction CitcomS mode measures temporal order approximately
1.01 for nonuniform diffusion on a fixed mesh. The result is reproduced on
tiny straight-sided triangles and tetrahedra without Stokes, spherical
geometry, checkpointing, or A1.

UW3 agrees with the independently assembled recurrence to roundoff. The loss of
second-order behavior comes from applying only two lumped-inverse corrections
to a residual assembled with the consistent mass matrix, together with the
lumped startup rate. A constant scalar decay does not reveal this because
consistent and lumped mass act identically on a constant field.

This PR therefore preserves the CitcomS-compatible fixed-correction algorithm
but does not claim general second-order PDE accuracy for it. The separately
selected CN path is verified second order. The distinct time_integrator="pc_converged" option now provides that
second-order reference without silently changing the compatibility mode. It
uses lumped mass only as an iterative preconditioner and converges the full
consistent Petrov-Galerkin rate residual at startup and every timestep.
Non-convergence raises RuntimeError.

Corrected-mode check	Serial	Eight ranks
2-D temporal order	`2.0001`, `2.0000`	`2.0001`, `2.0000`
3-D temporal order	`2.0001`, `2.0000`	`2.0001`, `2.0000`
Maximum trapezoidal-map difference	`<5.2e-14`	`<5.2e-14`
Corrections/step at test `rtol=1e-12`	`48-81`	`48-81`

The public default is rtol=1e-10. Repeated residual assembly is currently
more expensive than a reusable consistent-mass Krylov solve, so this remains
an accuracy/reference mode; fixed PC2 remains the Zhong production default.

Serial/MPI agreement from #692

The rotating-Gaussian comparison from #692, now included here, uses the same mesh
file, timestep sequence, and unchanged 1e-8 serial/MPI threshold.

Cell size	Cells	Before: serial/MPI difference	After: serial/MPI difference	Serial/MPI L2 error after
1/4	162	`7.00e-7`	`3.90e-11`	7.86% / 7.86%
1/8	614	`8.47e-9`	`2.01e-11`	3.02% / 3.02%
1/16	2,398	`2.46e-9`	`1.87e-12`	2.86% / 2.86%

The 1/4 absolute spatial error remains above the existing 5% accuracy bound;
only its partition dependence is fixed. No accuracy tolerance was relaxed.

The own-cell geometry oracle was also checked before and after affine
deformation on triangles, tetrahedra, quadrilaterals, and hexahedra. All four
geometries failed the old implementation in serial and MPI and agree with the
independent vertex-coordinate calculation to the existing 1e-12 tolerance
after the fix.

Restart and memory validation

Fresh-process tests write, exit, construct a new interpreter, restore, and
continue PC2, converged PC, CN, and BDF2 on a 370-tetrahedron unit cube. The uninterrupted run
is the oracle.

Restart run	Result	Maximum continued-field difference
Serial workers	4 passed	PC2 and converged PC exact; CN `5.55e-16`; BDF2 `8.88e-16`
Eight-rank workers	4 passed	same bounds

The default Level 2 regression runs PC2, converged PC, CN, and BDF2 on
triangles and tetrahedra for eight updates and checks stable PETSc/vector/workspace
identities, finite temperature, and physical bounds. The longer RSS test is
an opt-in Level 3 soak selected with UW_RUN_SUPG_MEMORY_SOAK=1; it retains
the 200 updates, excludes the first 40 from growth, and checks workspace
identity without forced garbage collection.

Run	Largest per-rank RSS growth	Largest late slope	Result
Serial	5.53 MiB	0.007760 MiB/step	6 passed
Eight ranks	1.34 MiB	0.010365 MiB/step	6 passed per rank

Acceptance is <16 MiB/rank after warm-up and <0.05 MiB/step late slope.
These are bounded regressions on small meshes, not proof of leak-free behavior
for every production mesh. No new memory-management workaround was required
after the shared implementation reused its workspaces. The table records the
completed full soak validation; routine CI executes only the fast lifecycle
test.

Validation on this review branch

isolated amr-dev worktree build: passed;
deprecated-pattern scanner and git diff --check: passed;
Level 1: 1,693 passed, 36 skipped, 2 expected xfails in 4m34s;
focused serial SUPG/geometry suite: 84 tests passed before the three
old-base supervisor failures; all three fresh-process methods passed in a
separate rerun after the compatibility gate (53s);
focused eight-rank solver/geometry/analytical suite: 31 passed per rank;
unified eight-rank rerun: 5 passed and 5 capability/baseline skips per rank;
default workspace-reuse test: 6 passed/6 soak skips in 16.47s serial and
35.03s at eight ranks; and
representative opt-in 200-step PC2 triangle soak: passed in 10.83s;
corrected-mode API/non-convergence checks: 7 passed in 5.57s;
corrected-mode fixed-mesh accuracy: 2 passed in 33.31s; and
corrected-mode eight-rank accuracy/workspace: 2 passed per rank in 57.68s
and 17.58s, respectively.

After cherry-picking the complete #691/#692 stacks and rebuilding this exact
branch, the focused serial cell-size, Nitsche, and SUPG residual suite passed
24 tests in 23.39s. A combined eight-rank batch passed 7 tests per rank in
41.50s, including the empty-partition mesh sequence, cell-by-cell size
comparison, default-Nitsche path, and the SUPG partition-independence test that
failed in the previous upstream run. No numerical tolerance was weakened.

The target branch predates two changes already merged into development:

Separate reconstructed and in-place PETSc checkpoint reloads #674 supplies exact same-layout distributed PETSc field reloads; and
Supervise every parallel batch: bound a collective hang and name the rank that caused it #678 supplies bounded MPI descendant supervision.

Tests detect those capabilities. On this old target, only MPI disk restart is
skipped and singleton restart runs directly with a timeout. A temporary merge of
current upstream/development and the exact #689 head (f41bcd2f) was built
without source conflicts. Supervised fresh-process restart then passed for PC2,
converged PC, CN, and BDF2 on eight ranks: 4 passed in 94.59 s. This verifies
the full paths while keeping the 924 lines already reviewed in #674/#678 out of
this PR.

Detailed experiment records and raw-result summaries are maintained in the
benchmark documentation,
especially the SUPG unification, analytical PC2, diffusion-time, restart,
memory, MPI-stall, and #687 cell-size reports.

Scope boundaries

No Stokes, geoid, reaction projection, level-set, or Navier-Stokes changes.
Generic mesh changes are limited to the focused Fix cell-family detection on empty MPI ranks #691/Make mesh.cell_size() local and partition independent #692 fixes and their
serial/MPI regressions.
No dependency or Pixi lock changes.
No solver tolerance changes outside accuracy-focused tests.
No claim that a small transport test replaces long A1 production acceptance.
Gadi A1 production trajectories remain benchmark work after this solver PR.

Checklist

Underworld development team with AI support from Claude Code

Use one public AdvDiffusionSUPG class and one F0/F1 assembly for implicit CN/BE/BDF and explicit CitcomS updates. Preserve the P1 positive lumped mass, gamma=0.5 two-correction update, directional tau and conservative timestep estimate, cached PETSc vectors and geometry. Replace the former implementation with compatibility imports; no duplicate solver remains. Register timestep-estimator and integrator snapshot state, omit unused CitcomS DDt history, retain explicit BDF selection and custom tau, and validate incompatible settings. Add frozen-source triangle/tetrahedron equivalence and in-memory/disk restart tests. Replace the MPI Gaussian cross-host golden value with a same-host serial reference without loosening the threshold. Validation: rebuilt Mac worktree; 25 unified/API tests passed, 13 residual tests passed, seven MPI tests passed per rank including the Gaussian regression, and nine expanded two-rank migration/snapshot tests passed per rank. Style gate and diff whitespace checks passed. Gadi coupled benchmarks and memory gate are still pending; this is not a production-acceptance claim. Underworld development team with AI support from OpenAI Codex.

Select backward Euler explicitly for quasi-steady manufactured and high-Peclet tests. Check the public generic tau expression instead of the removed P0 implementation detail. Compare spherical serial/MPI values on the same host and cached triangulation; keep manufactured convergence as the absolute accuracy gate. Apply the same near-machine-precision implicit replay tolerance as the unified snapshot tests. Syntax and whitespace checks pass; Gadi numerical validation remains pending.

The Gadi eight-rank migration gate failed before any timestep because the tiny triangle fixture leaves unsupported local simplex layouts on some ranks. Propagate layout errors to every rank before entering timestep reductions. Increase the equivalence fixture resolution and add a separate empty-partition rejection regression. Preserve numerical algorithms and tolerances. Rebuilt locally; Gadi validation pending.

Gadi preserved snapshot fields exactly but a rebuilt CN solve differed by 3.45e-10 with the default 1e-9 Krylov tolerance. Tighten only the restart-test KSP/SNES tolerances and retain the near-machine-precision replay assertion. Production solver defaults are unchanged.

Eight-rank PC2 migration and field replay now pass. The CN rate differed by 2.04e-12 because taking a timestep derivative divides field roundoff by dt=0.003. Propagate the unchanged 2e-14 field comparison bound through max(abs(delta T))/dt rather than using an unrelated fixed rate tolerance. Snapshot restoration remains exact and production settings are unchanged.

Exercise the explicit P1 gamma=0.5 two-correction path against smooth published pulse solutions on triangles/tetrahedra, the existing rotating-Gaussian oracle, and exact radial diffusion in a spherical shell. Keep SUPG active for advection and prescribe velocity without Stokes. Bound finite-domain tails, use common fixed timesteps for spatial refinement, check absolute errors and convergence, and optionally retain compact HDF5 diagnostics for same-host serial/MPI comparison. Verify the spherical equation symbolically. Solver implementation unchanged. Syntax checks pass; numerical Gadi validation is pending.

Refine the spherical diffusion pair from 1/4-1/8 to 1/8-1/16 without relaxing the 8 percent absolute L2 criterion. Add fixed-1/8 dt, dt/2 and dt/4 trajectories, restoring identical T/Tdot/startup state and comparing full FE fields. Record input Gmsh SHA256 and short mesh filenames for strict same-mesh serial/MPI comparison. Reuse a focused advance/measurement helper instead of duplicating the diagnostic code. Solver implementation unchanged. Syntax checks pass; bounded Gadi numerical follow-up pending.

gthyagi · 2026-09-05T21:22:24Z

@lmoresi This PR is ready for a focused review once the current CI run completes. Two decisions remain intentionally open:

Generic mesh fixes: should e4d0eaab (collective cell-family inference on empty MPI ranks) and fa75557f (cell-local stabilization length, fixing mesh.cell_size() is partition-dependent: the per-cell radius comes from a kd-tree over the rank's centroids, so cells at partition boundaries get a different size on each rank count #687) remain in this integration PR, or should I move either fix to an independent PR before merge? Both have focused serial/MPI regressions and are required by the validation cases in their current form.
CitcomS PC2 contract: the fixed two-correction algorithm is preserved as a compatibility integrator. Isolated diffusion tests measure temporal order near 1.01, while UW3 Crank-Nicolson measures 2.00 against the same exact discrete mode. The documentation therefore does not claim second-order time accuracy for PC2. Is retaining that exact compatibility behavior as an optional public integrator the intended contract, or should it remain benchmark-internal pending a later converged-correction variant?

The former 200-step RSS soak is now opt-in (UW_RUN_SUPG_MEMORY_SOAK=1). Routine Level 2 CI uses six short lifecycle cases (three transport methods in 2-D and 3-D), while preserving the full soak for release/HPC validation.

Keep test 1077's default cellsize 1/8, numerical setup, 5% analytical bound and 1e-8 serial/MPI comparison unchanged. Expose only the internal helper's mesh size for the requested eight-rank 1/4, 1/8 and 1/16 Mac investigation after an empty-partition mesh. All three resolutions completed after mesh fix 7e17bec. Same-host serial/MPI errors agreed within 8.47e-9 at 1/8 and 2.46e-9 at 1/16. Preserve and document the independent 1/4 discrepancy of 7.00e-7; it persists with tighter solves on identical mesh hashes. Do not relax the original gate.

Add tiny 2D triangle and 3D tetrahedron tests with independent analytical P1 element matrices and exact semidiscrete eigenmode references. Compare every UW3 T/Tdot update against a closed-form two-correction map without using a fine numerical solution, Stokes, or A1. Demonstrate that consistent residual mass with lumped correction mass and two fixed corrections approaches (2I-D^-1 M)D^-1 K, with first-order timestep differences. Separate the startup-rate error using exactly solved CN and genuinely lumped-residual controls, which establish why uniform scalar decay is insufficient. Document the limitation without changing CitcomS semantics, solver defaults, or prior tolerances. Four tests pass on Mac serial (17.46 s) and eight MPI ranks (32.91 s); matching mesh hashes and update discrepancies below 4e-14. Style gate and whitespace checks pass.

Exercise the actual shared Eulerian CN solver on the same tiny triangle/tetrahedron meshes and timestep sequence as the PC2 diagnosis. Check the exact CN amplification map and second-order error against independently assembled generalized eigenmodes. Two tests pass in serial (26.61 s) and on eight MPI ranks (13.69 s), with order 2.00 in both dimensions. Finest relative errors are approximately 5.4e-9 and 5.7e-9. No Stokes/A1 run, numerical-method changes, or relaxed error tolerances.

A tiny three-process restart regression exposed a missing _h_cell field: implicit SUPG previously created its generic-tau geometry field only during residual construction, while snapshots require the same fields registered before loading. Materialize this existing dependency at solver construction, only for automatic generic tau; do not relax snapshot schema checks or change the time integration. Add independent full/write/resume worker tests for PC2, CN and BDF2 on a 370-tetrahedron mesh with varying timesteps and velocity. Require exact restored fields/state, then compare step-12 continuation with uninterrupted runs. The CN case demonstrably failed before the four-line constructor fix. After rebuild: serial 3 passed in 48.95 s; eight-rank workers 3 passed in 58.98 s. PC2 continuation is exact; CN/BDF2 fields agree within 8e-16. Update the user guide with the fresh-interpreter workflow. No Stokes/A1 run or checkpoint tolerance relaxation.

Reduce timestep-control changes, FMG comparison norms, and adapted-mesh pulse maxima globally instead of assuming every partition contains the Gaussian peak. Make equality and finite-value failures collective before subsequent solver calls. Keep the existing 1e-3 change, 1e-6 relative error, and 0.9-1.01 peak thresholds. No equations or solver tolerances change. The prior rank-local failures stranded peers in later PETSc collectives. Validation: all 48 API/residual/migration/diffusion checks pass on eight Mac ranks in 152.40 s. The three modified API tests also pass in the final 12-test serial lifecycle sequence (98.14 s). Global FMG discrepancy is 3.04e-10 with amplitude 0.96026 on eight ranks.

…checks Exercise PC2, CN and BDF2 for 200 changing-velocity updates on tiny triangles and tetrahedra, without Stokes, checkpoint output, reaction diagnostics or forced GC. Record per-rank current RSS after warm-up and require stable solver/vector handles and PC2 workspaces. Keep preset per-rank bounds of 16 MiB growth and 0.05 MiB/step late slope, with collective temperature-finiteness checks. Delegate fresh-process restart execution caps and descendant cleanup to the existing MPI supervisor instead of duplicating process-group termination. Document actual UW3 CN second-order discrete diffusion validation and the distinction between small lifecycle regression and production-scale acceptance. Validation: final serial API/restart/memory sequence 12 passed in 98.14 s; final eight-rank restart 3 passed in 60.59 s; final eight-rank memory 6 passed per rank in 60.56 s. Maximum resumed field discrepancy 8.89e-16, PC2 exact. Repeated memory checks pass unchanged limits; no universal zero-leak or second-order PC2 claim. Style gate and git diff --check pass.

Run singleton fresh-process phases directly when the target branch predates the MPI supervisor merged in development by underworldcode#678. Keep MPI restart validation conditional on that supervisor so parallel descendants remain bounded and diagnosable. This lets the SUPG feature branch validate PC2, CN, and BDF2 restart state without importing the unrelated 691-line supervisor change into this review. Underworld development team with AI support from Claude Code.

Detect the same-layout checkpoint API added by upstream underworldcode#674 before asserting MPI disk replay. Older transport-branch checkouts continue to test serial disk restore and MPI in-memory restore; rebased development checkouts automatically exercise the full distributed disk path. This keeps the SUPG PR focused while making its dependency on the already-merged checkpoint fix explicit. Underworld development team with AI support from Claude Code.

Keep six fast Level 2 workspace-reuse checks for PC2, CN, and BDF2 on triangles and tetrahedra. They run eight updates and retain the deterministic object-identity, finite-field, and boundedness assertions. Reclassify the 200-update RSS and late-slope regression as an opt-in Level 3 slow test selected with UW_RUN_SUPG_MEMORY_SOAK=1. Preserve its warm-up, sampling, thresholds, and six-case matrix, and document both execution paths. Validation: default serial 6 passed/6 skipped in 16.47 s wall; default eight-rank 6 passed/6 skipped per rank in 35.03 s wall; representative opt-in PC2 triangle soak passed in 10.83 s wall. Deprecated-pattern and whitespace checks pass. Underworld development team with AI support from Claude Code.

gthyagi · 2026-09-05T22:04:08Z

Follow-up: the two generic mesh fixes have now been removed from this PR and submitted independently:

Fix cell-family detection on empty MPI ranks #691: collective cell-family detection on empty MPI ranks;
Make mesh.cell_size() local and partition independent #692: own-cell, partition-independent Mesh.cell_size() (fixes mesh.cell_size() is partition-dependent: the per-cell radius comes from a kd-tree over the rank's centroids, so cells at partition boundaries get a different size on each rank count #687).

The #689 history was rewritten and force-pushed without either commit. A clean rebuild succeeded. Its broad serial SUPG suite then produced 73 passed, 9 expected skips, and one expected prerequisite failure: without #692, the existing high-Peclet SUPG error is 0.13349, above the unchanged 0.10290 threshold. I have documented #691/#692 as prerequisites rather than weakening that assertion.

The earlier request to decide whether the generic fixes should stay in #689 is therefore resolved. The remaining review question is only the intended public contract for fixed two-correction PC2 and its documented temporal-order limitation.

Preserve the fixed two-correction CitcomS compatibility mode while adding an explicitly selected pc_converged integrator. Iterate the full Petrov-Galerkin rate residual at startup and each timestep, using the lumped mass only as a reusable correction preconditioner and failing clearly when configured tolerances are not reached. Persist correction controls in solver snapshots, expose convergence diagnostics, and extend workspace and fresh-process restart coverage. Add independent finite-element diffusion tests proving second-order convergence and exact trapezoidal-map agreement in 2D/3D serial and MPI without relying on the Zhong A1 model. Document the mathematical distinction from fixed PC2 and the current repeated-residual performance cost.

gthyagi · 2026-09-06T11:49:28Z

Marked ready for review without changing the requested target: feature/eulerian-supg-transport.

The head is 17 commits ahead of that target and contains no unrelated target divergence. Generic mesh fixes remain in independent PRs #691 and #692 rather than being duplicated in this SUPG integration diff. The current red CI result is the known partition-dependent Mesh.cell_size() check fixed by #692; the pc_converged tests did not fail.

DMPlexIsSimplex returns false on an empty partition. Using that local value for coordinate FE construction mixed simplex and tensor bases across the same communicator, consuming different PETSc message tags. A subsequent mesh HDF5 labelsLoad then blocked in PetscSFSetUp_Basic/MPI_Waitall. Gather cell-family decisions from populated ranks before FE setup and use the same classification for element metadata. Preserve the constructor hint for an entirely empty mesh; do not change SUPG algorithms, solver tolerances or MPI providers. Add single-cell triangle/tet/quad/hex regressions that require empty ranks and subsequently load another mesh, checking P2 volume and boundary integrals. The triangle regression failed before the fix. The original SUPG migration/partition sequence plus all four regressions passed on eight Mac ranks: 15 tests in 26.05 s, 32.30 s including launcher, 2.74 GiB peak process-tree RSS. The pre-fix sequence hung and the mesh-only reproducer also hung. Gadi rerun remains pending.

…#687) Adapt only the mesh-size correction from lmoresi's 68e545f on feature/navier-stokes-supg; do not import Navier-Stokes or other branch changes. Cache _radii_own from current DM vertex coordinates and use it for mesh.cell_size(). Preserve the legacy kd-tree radius arrays and global timestep/mesh-motion consumers. Use coordinate-section offsets and the full vertex stratum so the own-cell RMS definition also handles hexahedra, which have eight vertices but six faces. Correct the field documentation and Nitsche mechanism tests for the new definition; retain physical solve tolerances and use the exact nearest-centroid <= own-centroid ordering instead of an arbitrary approximate-equality tolerance. Add a first-failing independent geometry/deformation regression for triangles, tetrahedra, quadrilaterals and hexahedra plus a regular-square analytical control. Before: four failures in serial and on eight ranks. After rebuild: 21 passed/one expected skip serial (22.90 s), 22 passed on eight ranks (40.45 s), covering Nitsche solves, radius accessors, frozen PC2 migration and memory/disk snapshots. Own-cell geometry error is zero in these tests; style and whitespace gates pass.

Rename the new per-cell geometric radius cache from _radii_own to _cell_radii so the name describes cell geometry rather than rank ownership. Update the focused Nitsche and deformation checks accordingly.\n\nAdd an enumerated parallel regression that gathers owned-cell centroid/radius pairs and compares the complete sorted table with a fresh single-rank run on the same cached Gmsh mesh. This directly guards the rank-count-independence claim at np=2, np=4 and np=8 instead of relying only on within-rank geometric identities.\n\nValidated locally with 9 focused serial tests and the new MPI test at 2, 4 and 8 ranks.

Remove the local_h=False workaround from the boundary-normal MPI regression now that Mesh.cell_size() is partition independent. The test again exercises the public local_h=True default and compares its Nitsche solve with a fresh serial process.\n\nRecord the user-visible consequence in the development changelog: the rank-local centroid kd-tree moved the default Nitsche velocity answer by 6.6e-3, while the cell-geometry replacement is identical cell by cell from one through eight ranks.\n\nValidated the focused Nitsche regression at 2, 4 and 8 Open MPI ranks (10.99 s, 7.31 s and 9.60 s respectively).

gthyagi · 2026-09-06T12:15:34Z

PR #689 now includes the complete commit stacks from #691 and #692 as explicit cherry-picks, without retargeting the PR or importing unrelated development commits.

Source	Original commit	Commit in #689
#691	`8e93fc17`	`0e90dcb1`
#692	`443ffb58`	`0f6b78fa`
#692	`6fde1ac9`	`accf259b`
#692	`e6eaac28`	`f41bcd2f`

Validation after rebuilding the combined branch:

Run	Result	Time
Focused serial cell-size, Nitsche and SUPG residual suite	24 passed	23.39 s
Combined 8-rank empty-partition, cell-size, SUPG partition and default-Nitsche batch	7 passed per rank	41.50 s

The 8-rank batch includes test_1077_advdiff_supg_parallel.py::test_error_is_partition_independent, which was the sole failure in the previous #689 CI run.

gthyagi · 2026-09-06T12:29:33Z

@lmoresi The remaining current-development integration gate is now complete.

I created a temporary merge of current upstream/development and the exact #689 head (f41bcd2f). The source merge was clean apart from retaining the current-development changelog side, the merged tree built successfully, and the supervised fresh-process disk-restart test passed all four public transport selections on eight MPI ranks:

Integration check	Result	Time
PC2, converged PC, CN, and BDF2 fresh-process restart, 8 ranks	4 passed	94.59 s

This exercises the exact same-layout field reload from #674 and MPI descendant supervision from #678 without duplicating either implementation in #689. The PR description and restart table now record this result. Upstream CI for the exact PR head is still running.

The remaining maintainer decision is the public API contract: preserve fixed two-correction time_integrator="citcoms" for Zhong/CitcomS compatibility, with its documented first-order nonuniform-diffusion limitation, and expose time_integrator="pc_converged" as the separately selected second-order reference. Please confirm whether that split is the intended public contract.

gthyagi · 2026-09-06T13:13:06Z

Final author-side gate update: upstream CI passed for the exact PR head. The PR is now clean and mergeable, and every author-action checklist item is complete. The requested maintainer decision on the public PC2 API remains pending review.

gthyagi added 7 commits September 6, 2026 02:07

gthyagi marked this pull request as ready for review September 5, 2026 21:22

gthyagi requested a review from lmoresi as a code owner September 5, 2026 21:22

This was referenced Sep 5, 2026

Fix cell-family detection on empty MPI ranks #691

Merged

Make mesh.cell_size() local and partition independent #692

Merged

gthyagi added 9 commits September 6, 2026 07:35

gthyagi force-pushed the feature/zhong2008-supg-integration branch from b739d68 to 048ee09 Compare September 5, 2026 22:03

gthyagi marked this pull request as draft September 5, 2026 22:09

gthyagi marked this pull request as ready for review September 6, 2026 11:49

gthyagi added 3 commits September 6, 2026 22:13

Navigation Menu

Uh oh!

Integrate CitcomS predictor-corrector into unified Eulerian SUPG - #689

Integrate CitcomS predictor-corrector into unified Eulerian SUPG#689
gthyagi wants to merge 21 commits into
underworldcode:feature/eulerian-supg-transportfrom
gthyagi:feature/zhong2008-supg-integration

gthyagi commented Sep 5, 2026 •

edited

Loading

Uh oh!

gthyagi commented Sep 5, 2026

Uh oh!

gthyagi commented Sep 5, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gthyagi commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why

Public numerical contract

Bugs found during validation

Independent accuracy checks

Important PC2 limitation

Serial/MPI agreement from #692

Restart and memory validation

Validation on this review branch

Scope boundaries

Checklist

Uh oh!

gthyagi commented Sep 5, 2026

Uh oh!

gthyagi commented Sep 5, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

gthyagi commented Sep 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

gthyagi commented Sep 5, 2026 •

edited

Loading