Fault networks in parallel: co-located multigrid tail, gather-safe pair diagnostics - #662
Merged
Merged
Conversation
…h walls The zone outcrop no longer needs an axis-aligned wall. The cap over the bowl is re-triangulated PER COPLANAR BOUNDARY REGION, each piece meshed flat in its region's own plane, so the cap stays exactly on the faceted surface and volume conservation survives. Where the band crosses a crease the two sides segment the line differently (mesh vertices against assembly nodes), so a 1-D overlay merges both node sets by arc coordinate and keeps each elementary sub-segment for the side(s) whose bowl covers it and the band does not; a one-side segment must be a whole mesh edge, because a cavity-shell face matches it. The frame rule (_outcrop_frame_3d) separates the two notions the third dimension forces apart. Where the cavity may OPEN is by SMOOTH WALL — coplanar regions grouped across low-dihedral creases — so a faceted sphere's bowl legitimately spills onto facets beside the band's own while a box's other walls stay refused. What the carve may DELETE is by shape: band-covered vertices, single-region interiors, and interior vertices of straight creases between touched regions; a vertex where three or more regions meet is the domain's shape and is protected. A protected vertex whose whole cell star drops is not stranded: it is a collar node, and the gap fill's tets reference it — that is how a curved boundary keeps its faceting through the surgery. Wall labels are restored per removed face (the 2-D per-segment rule one level up): each new wall triangle takes the labels of the removed face it lies on, so a bowl spanning several walls restores each wall's own. The Euler gate now demands CONSERVATION of the input's Euler number rather than 1 — a spherical shell is S^2 x I, Euler 2, and refused before for its topology, not for any defect. The same assumption in place_sheet and remove_embedded is marked TODO(BUG). _trace_wall_code and its refusals are deleted; the box path runs through the general machinery and the box oracle tests are unchanged. Driven on a rotated box (no wall axis-aligned), a band across the Top/Front box edge (trace on both walls), and a spherical shell outer surface (75 trace facets, 6 vertices removed): volume conserved to 1e-12 in each. Underworld development team with AI support from Claude Code
…n a sphere Three configurations, each the failure mode of a different assumption the box-framed cap made: a rotated box (no wall axis-aligned; the single-region collar), a band across the Top/Front box edge (the crease overlay conforms the two sides' differing segmentations, and each wall's labels are restored on its own side), and a spherical shell outer surface (every facet its own region, every vertex protected faceting — the case the trace design exists for). Each runs its negative control first — the identical census on an interior twin counts no trace — and each ends in the P2 Poisson oracle, exact for a quadratic through the zone. Volume conserved to 1e-12 throughout; the shell's Euler number 2 comes through the conservation gate. The parallel file gains the rotated-box outcrop at np>=2 with the same mesh and patch as the serial test: the info dict is identical on every rank and the trace count matches the serial value (28 facets at np=1, 2 and 4), so the general path is partition-independent. No boundary face is left without a wall label at any rank count. The spherical control sits mid-gap of a 0.75-thick shell: a one-cell chain from a victim corner spans h, so an interior zone needs ~(clearance + 1) * h to spare on both sides — thinner shells refuse interior zones at this resolution by the carve's own clearance gate. Underworld development team with AI support from Claude Code
The collar-vs-bowl consistency check raised on the surgery rank inside the relabel block, which is outside any try — a hang at np>=2. It now sets the block's failure and flows through the allgather like every other refusal there. Underworld development team with AI support from Claude Code
The same gate place_thin_volume already fixed: demanding global Euler number 1 encodes a ball-topology domain, and a spherical shell — S^2 x I, Euler 2 — was refused for its topology rather than for any defect of the surgery. Both gates now compare against the input mesh's own number. Pinned by a regression test: a sheet embeds mid-gap in a spherical shell and its removal clears the label again, both passing their volume and conservation gates. Underworld development team with AI support from Claude Code
The gate catches unmeshed solids — an O(1) relative defect — but its reference, OCC's getMass on the clipped boolean, is only accurate to ~5e-7 relative when the domain tool carries many faces (measured on a 16.6k-facet adapted spherical boundary: the honest mesh volume exceeds the reported CAD mass before any snap runs). At 1e-9 the gate refused correct assemblies; it now allows 1e-6, far above the kernel noise and far below any real missing-solid defect. Underworld development team with AI support from Claude Code
OCC's boolean leaves clipped nodes up to ~4e-7 off the tool's own planes on O(1) geometry against a many-faceted tool; the 1e-9 snap missed them, so a crease-crossing node of the band outline was not recognised as on-crease and the collar meshed a 2-metre sliver beside the crease (measured on a 1000 km megathrust against an adapted spherical boundary; the 2-D fill then refused with moved nodes). The snap tolerance now sits above that noise and below any layer mesh size, and candidates are masked by distance to the boundary FACETS first — the planes are infinite, and at this tolerance every point in space is near some plane of a many-faceted tool. Underworld development team with AI support from Claude Code
Two mechanisms a 1000 km outcrop band forced, plus the diagnostic that found them. _collapse_boundary_imprints_3d is the 2-D imprint collapse one dimension up: where the band outline grazes a boundary vertex (measured: a 2.4 m gap the collar meshed as a sliver and the fill refused as moved nodes), the outline is rerouted THROUGH the vertex — the nearest outline node moves onto it, or the outline edge splits at it with every incident tetrahedron bisected. A move or split must keep every incident cell's volume healthy, else the vertex is skipped and keeps its sliver (the status quo, not a defect); the band still tiles the same faceted surface, so the domain's shape is untouched. An outline edge lying ALONG a crease is no longer refused when the cavity covers both sides: the band reaches the crease there, so the edge bounds the collar piece on the FAR side of the crease, not the owning band triangle's own region. The refusal remains for a band bounded by a crease with no bowl beyond it. A collar piece that fails to mesh now names its pinch — the thinnest node-to-segment gap and the node kinds — which is what separated this sliver class from the OCC placement noise the snap fix covers. Known limit, measured and deliberately not papered over: refining a SPHERICAL boundary projects new facet vertices onto the true sphere, and the creases between the resulting nearly-coplanar sub-facets are laterally fuzzy at (placement noise)/sin(theta) — wider than the band's own feature spacing, which defeats the crease overlay. A widened, fuzz-aware overlay tolerance was tried and withdrawn: it traded the sliver for misclassified chains on the honest-crease tests. Outcrops on adapt-refined spherical boundaries stay refused by the fill's own gates until the crease representation is rethought. Underworld development team with AI support from Claude Code
The sheet path joins the general boundary machinery: one clip, one frame, one collar (maintainer ruling 2026-08-18 — multiple placement paths are themselves the defect). The Sutherland-Hodgman box clip and the wall-code frame are deleted. The discrete clip primitive cuts the authored triangulation against the gathered boundary complex directly: per-component outward orientation (a shell's inner surface signs opposite to its outer), signed-distance classification, sequential cuts by the COPLANAR REGIONS' planes — a box wall cuts as one plane however it is faceted, or the two triangles sharing a cut edge key their cuts by different facets and the sheet tears (measured: a duplicated node on the box top wall). Side cuts are re-derived from the original sheet edge's endpoints and interned by (edge, region), so neighbours whose sides are truncated differently produce the bitwise-identical node (measured: ~1e-17 duplicate pairs on the sphere without it). A crossing that can touch a locally concave crease (an inner boundary) refuses loudly — the polyline cut is not built — and every kept node is gated inside the domain on exit. The carve takes the frame's two masks (open_deletable / open_near) like the volume's carve, with the same protected-collar-node rescue; the outcrop frame accepts a trace CHAIN of edges as the footprint alongside a band of triangles; and the collar embeds the chain through its pieces instead of cutting a hole — chain nodes at crease crossings land ON the crease and enter the 1-D overlay as 'a' nodes, the runs between crossings embed per region (_gmsh_fill_2d now takes several polylines), free ends as gmsh's ordinary free-end embed. A trace edge along a crease, or off the bowl, refuses with the reason. The overlay's local loop variable is renamed crease_chain — it shadowed the new parameter, which made the volume path read a stale trace. The trace chain's edges carry <label>_trace in the result, the wall's labels are restored per removed face (the volume path's rule), and the counts are gated collectively. Box oracle unchanged (test_0854, 8/8; area matches Sutherland-Hodgman bit-for-bit in the spike); the sphere outcrop that could not run at all now embeds with volume conserved and the P2 oracle exact (test_0860); trace counts identical at np=1/2/4 (ptest_0854). Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
The hybrid-seam study measured the defect this guards against: a split surface terminating against something that cannot slip ends as a free crack tip, slip pins to zero, and the composite is worse than either pure representation (74.6% vs 100/105%). The outcrop is that join one level up, so the acceptance test is kinematic — measured slip > 0 at the trace against a pinned control — not mesh gates alone. The placement half asserts today: every trace edge on the wall AND bounding a labelled fault face, the wall's labels restored beside it. The kinematic half attempts split_along_label_3d through the wall and SKIPS at its daylighting refusal — the body below the gate is the acceptance criterion, ready to run when feature/fault-split-node learns to duplicate the trace chain. Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
The boundary snap's tolerance was raised to 1e-6 to cover OCC's ~4e-7 boolean placement noise (34ea192), but the test's untouched-control point stayed at 2e-9 — inside the new tolerance, so it snapped and the test failed. The control moves to 2e-5: above the tolerance, below any real feature scale. A branch push does not trigger CI, which is how this shipped red. Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
Ruling (2026-08-18): split-node faults do not split through the surface. They stop an element or two below, blind, with a damaged region above carrying the deformation to the surface — the junction lessons applied: a contact tip inside weak material is free, a tip at the weak zone's edge is pinned, and an abutting composite is worse than either pure form. This supersedes the split-through-the-wall acceptance this file first carried; the gated body that waited on a daylighting split is removed. The kinematic acceptance turns the listric handover rule vertical, against a FREE surface, and is validated by running (probe in ~/+Simulations/blind_fault_surface_expression/): a blind frictionless split fault two cells under the surface, solved bare / damage-abutting / damage-enclosing, orders strictly on near-tip slip (0.45 / 0.65 / 0.80) and on surface localization (37% / 44% / 50%). The abutting case is the deliberate negative control — the seam defect the overlap margin exists to avoid. The structural test keeps the placement contract: the trace chain labelled through to the wall, now as the surface LOCATOR for the damage region rather than a split path. Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
…boundary The ruling's other half: the job is still to figure out the intersection so the fault can stop BEFORE it hits the surface — it just never meshes or splits all the way there. place_sheet(..., setback=d) clips the sheet against the boundary offset INWARD by d (each coplanar region's plane shifted along its inward normal; on a curved boundary the shifted faceted planes, within a sagitta of the true offset): the placed sheet arrives BLIND, its rim strictly interior, so split_along_label_3d takes it as it stands — the daylighting refusal never fires. The would-be intersection with the true boundary is still computed from the unclipped input and returned as info['surface_trace'] — the locator for the damage region above. The clip's output is made split-safe by construction: a cut corner polygon (two side-rim originals plus cut nodes) admits no triangulation without an all-rim face, which the split refuses however fine the sheet (measured: 3 such faces at every density tried). The face's longest interior edge is split at its midpoint — bisecting both sharing faces, rim edges (the trace included) untouched, no child worse-shaped than its parent. A centroid split was tried first and REJECTED: a centroid inside a sliver corner face drove the gap fill to 1e-23 cell volumes and a P2 error of 0.64 on the shell outcrop. test_0861 gains the end-to-end workflow test: a through-running sheet placed with setback=0.25 arrives with no trace on the wall, its shallowest vertex exactly a setback below it (the box's offset plane is exact), surface_trace on the true boundary, and the placed patch splits with the Plus side carrying every placed face. Box oracle and the outcrop suite unchanged (test_0854 8/8, test_0860 4/4, suite green, np=2/4). Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
…tor marches Physics-first ruling (2026-08-19): choose the fault's blind depth from the physics and adapt the resolution to meet it. Two library changes make that workable on the meshes it needs — locally refined shells whose boundaries snap to the true sphere. The clip's concave refusal becomes a MEASURED gate. A snapped multi-resolution boundary is genuinely non-convex where fine facets meet coarse chords, but the concavity DEPTH there is the sagitta mismatch — metres at Earth scale — while an inner boundary's is the facet size. The sequential plane clip over-cuts by at most that depth, so a crossing is allowed when the deepest concave crease among the near facets stays under 20% of the setback, and refused otherwise; setback zero (an outcrop, where cut nodes must land on the complex exactly) still refuses any concavity at all, as before. The surface-trace locator contours the boundary signed distance over the sheet's own triangulation (marching triangles): every crossing point interpolates on a sheet EDGE from that edge's two vertex distances, so the two triangles sharing it produce the identical point and the polyline chains exactly, with no tolerance welding, on ANY boundary — concave, graded, snapped. A direct triangle-triangle intersection was tried first and REJECTED by measurement: robust- geometry endpoint mismatches fragmented a 300 km trace into 148 components. Locator accuracy is the linear interpolant's, O(spacing^2/R) — metres, which is the locator contract. Verified end to end at Earth scale (probe: ~/+Simulations/spherical_slab_outcrop/megathrust_blind_adapted.py): a 19,768-cell shell adapts to 288,525 cells (proportional grading h <= 0.5*distance — required, or coarse deep-rim tets span to the wall and the carve refuses; edge_split engine — NVB's volume proxy leaves diameters ~3x coarser than the carve's reach rules read), and the megathrust places blind at 15 km = 2.5 local elements with the trace located. Suites unchanged: test_0854/0859/0860/0861 21/21, spike green. Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
…ulates The ruling (2026-08-19): the authored fault triangulation is DATA, at whatever spacing the source provided — but the embedded fault's resolution must match the mesh it cuts, or the verbatim embed forces sliver cells around every mismatched sheet triangle. place_sheet gains size= — the counterpart of place_thin_volume's size, closing the asymmetry the unification left: the clipped sheet's rim is compressed to its exact corners (a too-fine authored rim coarsens here), each straight run is resampled at the target, and the interior is re-meshed by gmsh in the sheet's own plane. Planar sheets only (a curved surface needs a parametric remesh, refused with the reason), and not for an outcropping sheet — its trace chain must stay on the boundary complex verbatim, so blind (setback) and interior placements only. The all-rim split-safety pass is factored out (_split_safe_triangulation) and applied to the resampled triangulation too: a gmsh planar mesh of a polygon produces corner faces with all three vertices on the rim just as the clip's corner polygons did. test_0861 gains the workflow test: a deliberately coarse 3x3 authored sheet placed blind with size=0.1 refines to gmsh-quality faces (min quality > 0.3 asserted), keeps the blind rim exactly on the offset plane, and splits with the Plus side carrying every face. Suites unchanged (0854/0859/0860/0861 22/22, spike green). Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
A reach-aware spatial hash (_reach_query: points binned per octave of their own threshold) and a culled distance (_sheet_distance_within: exact wherever it is below the per-point reach, sentinel above it) replace the whole-domain per-triangle sweeps at the placement's nine hot sites — the gather marks, the carve's victim distances, its straddle sweep and centroid rule, in the sheet and volume paths alike. Per-point reaches matter: the thresholds scale with the local cell size, so a single cutoff cannot cull a graded mesh. Every caller thresholds where it reads the distance, so decisions are identical — verified: the placed mesh is bit-identical on the 452k-cell middle-ground benchmark, and place_sheet drops from 555 s to 67 s (the 446 s _sheet_distance share to under a second; the sewn rebuild is now the placement's largest piece at 30 s). Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
Three repeated whole-mesh walks removed from the adapt loop, none changing a single cell of the output (bit-identical child on the 452k-cell benchmark): - cell_diameters(return_tables=True) hands its cell-edge and edge-length tables to bisect_longest_edges, which was re-deriving both in the same pass — the per-cell closure walk is the loop's dominant Python cost and was paid twice per pass (309 sweeps -> 155). - cell_diameters' per-cell max is vectorised (a simplex has a fixed edge count, so the ragged list stacks). - The MG level selection re-measured every retained generation's 5th-percentile diameter at the end — 76 more topology walks — when the marking pass had just computed exactly those numbers; they are cached per-dm and passed as resolution_hint. Adapt on the benchmark: 658 s -> 551 s. The remaining ledger is the engine surgery recorded on #610 — per-pass clone + label writes inside the split call (242 s), MG parent/prolongation maps built per pass (107 s), independent-edge selection (54 s) — the plan-in-numpy / apply-back-to-back and multi-edge-template items. Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
The subsampler was already discarding every per-pass parent map whose span covered more than one pass — nearly all of them — so ~80 s of the benchmark's 107 s of map building produced maps that were thrown away, while the retained multi-pass levels got None and the any-degree transfer fell back to the geometric builder. The engine loop now records nothing; the subsampler derives parents for the 3-4 RETAINED pairs from the composed vertex transfer (nested_cell_parents is topological through the transfer, and a descendant's referenced coarse vertices are corners of its ancestor at any depth), so the deferral is also an upgrade: multi-pass levels now carry exact parents. Measured on the 452k benchmark, bit-identical child: parent-map cost 86 s (x75) -> 7.8 s (x9); adapt 551 -> 473 s. The bisect interior was also split with instrumentation: of its 242 s, the native transform's setUp+apply is 187 s (~2.5 s per application — the per-pass C cost is honest, the x76 pass count is the waste) and the independence pruning 54 s; clone and label writes are negligible. The remaining #610 levers are therefore pass-count reduction (multi-edge templates, C-side) and the plan-in-numpy replacement of the per-pass Python (~170 s). Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
The metric's far clip must sit above the base mesh's measured cell DIAMETERS, not its nominal gmsh cellSize: diameters run 1.2-2.5x the target edge length, edge_split marks on diameter, and a clip below them refines the entire domain once — wasted cells AND a measured de-conditioning of the far field (median shape quality 0.372 -> 0.295), which every later adapt and solve inherits. Found by eye in the viewer (scattered one-level patches across the globe), confirmed by measurement, fixed by h_far >= 1.05 * cell_diameters(base.dm).max(). Underworld development team with AI support from Claude Code Claude-Session: https://claude.ai/code/session_01WDKP73LD3dCFQSNgEXbbw9
…620) The zone walk went through _cells_on_edge, an EDGE walk: correct for a 2-D facet (an edge), one level too low for a 3-D facet (a face), where support-of-support lands on nothing and the zone comes back all-False, silently. The method had only ever seen 2-D meshes. Facets now take getSupport directly — the cells, in any dimension — and the edge walk is kept for labelled edges (a trace chain). Fixes #620. Underworld development team with AI support from Claude Code
The Mesh-level form of place_sheet, mirroring add_conforming_surface: the sheet is cut at the finest level ONLY, and the child inherits this mesh plus everything below it as its coarse multigrid tail — the coarse levels carry neither the cut nor the label, and do not need to (the Galerkin coarse operators are formed from the fine operator; measured in add_conforming_surface's note). The adoption — wrap, bookkeeping, and the "a cut earns a level only if it is genuinely finer" subsample — is extracted from add_conforming_surface into _adopt_cut_child and shared, so the two dimensions cannot drift. The method takes explicit (points, triangles, name): a sheet is DATA — a slab model, an authored parameter-space triangulation — whose connectivity must be embedded verbatim, and FaultSurface re-derives its triangulation so it cannot carry an authored one. setback (blind faults, surface_trace in child._surface_info), size (resample to match the mesh being cut) and clearance pass through to place_sheet. test_0862 mirrors the 2-D contract of test_0844: the tail composition and the replace-not-stack decision, the named boundary and its zone, the untouched base, chaining, the duplicate-name refusal, and a solve with a per-cell contrast across the sheet consuming the cut mesh as it stands. Underworld development team with AI support from Claude Code
_build_3d(mesher="place") placed each sheet at DM level and wrapped the result in a bare Mesh, so the adapt child's multigrid tail died at placement and every solve on the composed mesh fell back to algebraic multigrid — not by design, but because the handoff that add_conforming_surface performs in 2-D was never written here. The route now chains add_conforming_sheet, so each cut child inherits the hierarchy; the split at the end still forfeits it (see add_fault), but the intermediate cut mesh now carries the tail that split-aware multigrid would need. The recorded place-route pathology (solve 3850 s vs 125 s embed, on edge_split children) predates this handoff and wants re-measuring on top of it. Underworld development team with AI support from Claude Code
…rator health (#621) The 3850-vs-125 note said the composed chain's solve was pathological on adapt children. Measured again with the tail handoff in place: 27x the cells embed builds for the same nominal sizes, 41x the time — which is proportionate under 3-D Stokes scaling, with comparable per-cell cost, one nonlinear iteration, machine-zero leak and agreeing slip. The over-build is the route's own sizing (base at cellSize=h_far with refinement=1 puts the far field at h_far/2). Docstring restated; the sizing fix is #621. Companion measurements: #622 (custom-P V-cycles across a cut level cost 7x more than GAMG saves), #623 (fill volume drift on steep gradings). Underworld development team with AI support from Claude Code
inject_custom_mg re-ran build + install on every solve: a duplicate Jacobian assembly (done only to make the fieldsplit reachable), a PCMG reset + Galerkin setup, and a from-scratch geometric transfer build — measured at 55 s of every repeat Stokes solve on an 85k-cell cut child, while the transfers depend only on the meshes and PETSc re-Galerkins changed operator values at PCSetUp by itself. The guard checks the LIVE PC (the managed block exists, is a PCMG, carries the hierarchy's level count) against a marker written at install; any doubt re-installs — the cost of a wrong False is one redundant install, the cost of a wrong True would be a solve on a stale PC. Measured (place_route_health benchmark): repeat-solve overhead 55 s -> 0; a genuine solve (operator values changed) keeps mg:4its on the velocity block against gamg:46 and now comes in ahead on wall time as well (284.6 s vs 307.7 s with the fgmres outer). Underworld development team with AI support from Claude Code
…e sub-blocks but never the top (#624) SNES_Stokes_SaddlePt sets both fieldsplit sub-solves to fgmres but never pushed an outer ksp_type, so every saddle-point solve ran PETSc's default plain gmres around variable-iteration Krylov inner solves — a non-constant preconditioner under which standard GMRES's recurrence does not hold, the same reasoning as the velocity-block FGMRES note (#147) one level up. Measured on a genuine 85k-cell contrast solve: 14,400 inner iterations / 543.9 s under gmres; 307.7 s under fgmres (gamg velocity), 284.6 s (guarded custom-P). Pushed as a managed option in both the constructor and the strategy setter, so an explicit user ksp_type still wins. test_0203's lower-bound test asserted a STRICT undercount on the first solve's velocity count — an artifact of left-preconditioned gmres, which burned one full preconditioner application before its first monitor fired. Right-preconditioned fgmres forms the unpreconditioned residual first, so the bound is now tight; the test asserts the contract (second >= first, plus the honesty flag) instead of the artifact. The remaining genuine-solve bottleneck is the pressure sub-solve at its 200-iteration gasm cap, silently unconverged — #625. Underworld development team with AI support from Claude Code
…), plus a zero-column repair for placed levels The "rbf" transfer builder assembled and solved the GLOBAL dense coarse-cloud kernel matrix and returned nnz/row == n_coarse — dense Galerkin coarse operators, no conditioning path, a rescue that could not scale (#429, agreed long ago and never landed; the interpolation note recorded it as "Not rewritten"). It now delegates to the standard kd-tree local linear-exact interpolator (kdtree.interpolation_matrix, order=1) — the same polyharmonic kernel and reproduction guarantees, sparse support, the interpolator the rest of the code uses. A row-wise kNN builder guarantees nonzeros per ROW, never per COLUMN (#424): on NON-NESTED level pairs — two independently placed meshes (#626), a relaxed child — a coarse DOF outside every fine stencil gives an empty column and a singular PtAP. The serial build loop now repairs such columns by nearest-fine-DOF injection (weight 1, same component), warns with the count, and still refuses if any column stays empty. These are preconditioner transfers, not the discretisation: an injected row costs iterations at worst, never correctness. First measured use: the split-finest-over-unsplit-band ribbon hierarchy (6 of ~50k columns repaired; converged, machine-zero leak, answer unchanged). Suites: test_1015 / test_1016 / test_0753 — 28 passed, 1 pre-existing conditional skip. Underworld development team with AI support from Claude Code
…622) The rotated custom-P path hardcoded coarse="svd" for the velocity PCMG because a free-slip enclosure's Galerkin-coarsened velocity block inherits the rigid-rotation null modes and redundant/LU zero-pivots on them. But nsp being non-None cannot discriminate — it also carries the constant-PRESSURE mode (enclosed domains), which lives outside the velocity block — so a Dirichlet-walled problem with no rotation modes still paid a DENSE SVD factorisation of a 3-D P2 coarse level. _rotated_nullspace now records the count of VERIFIED rotation modes on the solver, and the coarse solve is SVD only when that count is nonzero; the fallback (count unknown) keeps SVD whenever nsp exists, so free-slip shells are untouched. Measured on the split-fault contact benchmark: the SVD was NOT the dominant cost (the per-cycle weight is the level-smoothing footprint of a localized-refinement hierarchy — the background rides at full size on every band level — recorded on #622); the gate is correctness of configuration, not the headline saving. Suites: 0844 / 0845 / 1015 / 1016 / 0203 — 63 passed. Underworld development team with AI support from Claude Code
The multigrid transfer needs more than DOF coordinates. To evaluate a coarse Lagrange basis inside a parent cell it must know WHICH rows of _get_coords_for_basis are that cell's own nodes -- and the section that answers that lives on a coordinate DM _get_coords_for_basis builds and then destroys. _basis_coordinate_dm is extracted from _get_coords_for_basis (second occurrence of the same construction) so both read the same layout, and _cell_node_indices walks that DM's local section over each cell's transitive closure. Degree 3, where an edge carries two DOFs, falls out of the offset/ndof expansion; a discontinuous space hangs every DOF off its own cell and needs no special case. Non-simplex meshes are refused explicitly: a tensor-product Q_k cell carries (k+1)^dim nodes, so the total-degree monomial basis the transfer builds would not be square against them. Cached beside _coord_array and cleared with it -- both describe the same node layout, and it is the DS re-creation, not node motion, that invalidates either. Tested geometrically rather than against itself: every node a cell claims must lie inside that cell, checked with barycentric coordinates built from the cell's vertices, at P1/P2/P3 in 2D and 3D for both continuities. A wrong section offset points at another cell's node, which is somewhere else in the mesh, so it cannot pass. Also asserted: node counts equal C(k+d, d), continuous rows are all covered, discontinuous rows partition cell-block-contiguously (the layout _build_kd_tree_index_DS already assumes), and Q_k is refused. Underworld development team with AI support from Claude Code
…geometric transfers, and FAC patch smoothing (#629) Item 1 of the #629 program, plus the fill mechanism it exposed. A level pair that is a native refine() pair (tagged by _coarse_level_meshes via hierarchy-slot sentinels) now gets the EXACT nested prolongation at any polynomial degree: parent cells recovered topologically from the two DMs, weights by the dual-basis identity W = B M^-1 under the parent's affine pullback (the #425 construction; verified exact to 1e-15 at P1/P2 in 2D and 3D). PETSc's DMCreateInterpolation general path was measured NOT to be the embedding (row sums to 1.375, quadratic reproduction error 1e-2) and is not used. The larger density win turned out to sit beside it: the point-located builders emit ~1e-16 junk weights that are STRUCTURAL nonzeros, and Galerkin RAP fills by structure, compounding level over level. Dropping them (_drop_structural_zeros) collapsed the split-contact benchmark's Galerkin chain from 319/481/265/90 nnz/row to 138/119/173/90 — the item-1 acceptance band — and the warm full-tail contact solve from 97 s to 52 s at bit-identical answers (slip 0.1509, leak 0). Full-tail iterations reproduce the recorded 6-7 under the robust smoother (the 6-its dial row was gmres/4; the script's fast row runs 12). Item 2: FAC/MLAT patch-restricted smoothing. Each level's patch is read off its own transfer (identity row + bit-coincident node = background, per NODE so the rotated path's per-node Q cannot disturb it; slit duplicates count as patch; halo = one coarse-cell layer through the transfer graph), stashed on the hierarchy, and consumed by _configure_pcmg: the level smoother PC becomes ASM with that single subdomain. Three configuration findings are load-bearing: * BASIC, not restricted, ASM: discarding the halo correction stalls the outer KSP at 80-375 iterations where basic runs 6 against a whole-level baseline of 4 (banded Poisson, 4-level tail). * The subdomain solve is SOR, the patch twin of the whole-level smoother: PCASM's default ILU-0 takes a NUMERIC_ZEROPIVOT on the rotated Galerkin patch block (min |diag| 8e-5 near the constraint; PC_FAILED -11 before the first iteration). * The rotated path must force the FAC levels' smoother setup before its options-DB cleanup: PCASM creates the sub-KSP lazily at first apply, after the keys are gone, so sub options silently never applied. A uniform pair (patch = everything) declines to whole-level smoothing. Measured on the split-contact ribbon: correct answers with rotation and contact, but a net cost at THIS toy's proportions — the ribbon band is 66-82% of every refined level's DOFs, so there is no background to save and the ASM gather/scatter is overhead (warm 60 vs 52 s). The FAC win requires production proportions (thin patch in a large domain), which is where the item-3 contrast test goes next. The UW_FAC_* / UW_CUSTOM_MG_* environment knobs are TODO(MEASURE)-marked A/B affordances for that campaign. Tests: test_1017_nested_native_and_fac.py (exactness by degree-k reproduction, tag discipline, split classification with a geometric oracle, end-to-end ASM solve vs GAMG); test_1015/1016/0753/0846/1014/ 1017/1018 suites green. Underworld development team with AI support from Claude Code
…tem 3 diagnosis) The contrast sweep's FMG degradation (velocity 7 -> 10 -> 36 its over 1e0 -> 1e4, where constant V-cycles are the expectation) is DIAGNOSED and FIXED at the smoother: * Level ablation at 1e4: full/no_mid/two tails converge in 20/19/24 its — the coarse corrections contribute nothing under zonal contrast; the weak-band modes are invisible to every coarse space, including the band-carrying mid level. * Doubling the smoother (gmres/8 vs /4) exactly halves the count (20 -> 10): convergence is proportional to total fine-level smoothing work — the signature of smoother-limited band modes. * FAC with a DIRECT subdomain solve (sub_pc lu) on the patch — which contains the weak zone — collapses the count to 2 its at 1e4 and 3 at 1e6: V-cycle constancy across six decades of contrast, below the flat-viscosity count (7). Two supporting changes: factorization sub-solvers get sub_pc_factor_shift_type=nonzero (the rotated Galerkin patch block carries near-zero pivots from the constraint-zeroed transfer rows — unshifted LU takes NUMERIC_ZEROPIVOT even as an exact factorization), and UW_MG_SMOOTH_ITS joins the TODO(MEASURE) campaign knobs. Remaining engineering to make the strong-patch cycle cheap: shrink the subdomain from the whole refined patch to the weak zone + halo, and reuse factors across cycles; the wall-time gate at contrast stays the pressure gasm cap (#625). Underworld development team with AI support from Claude Code
…duplicated nodes, not the refinement patch (#629) The refinement patch conflates the smooth refined bulk (well served by ordinary multigrid) with the fault zone (the only set needing the strong solve). The slit mode keys the subdomain on the split-duplicated nodes with operator-sparsity overlap as the halo — proportional to the fault trace, not the refinement. Validated on the 2-D line rig: constant 6-7 velocity iterations across six decades of viscosity contrast on a 5%% subdomain, against a GAMG control that degrades to its iteration cap. Underworld development team with AI support from Claude Code
…chanism) Measured on the parity-tuned [L0, L1] + finest curved-ladder stack, warm repeats, leak machine-zero in every arm: gamma=1 shortens the pressure loop 22 -> 17 with the velocity count unmoved (7), taking the warm solve to 19.1 s against GAMG's 23.0 — FMG ahead outright at flat viscosity. gamma=10 over-stiffens the velocity block (7 -> 12) and reverses the gain. No Schur plumbing was needed: the penalty is viscosity-scaled by construction, so penalty/mu is uniform and the 1/K pressure-mass preconditioner remains spectrally correct. Caveats recorded with the result: under penalty the recovered p is the Lagrange multiplier (p_mech = p - lambda*mu*div_u for any pressure-dependent rheology), and derived quantities drift with gamma (peak slip 1% at gamma=1, 3% at gamma=10 at rig resolution — the same bias family as #633's vertex-sampled dynamic topography). gamma=1 is the sanctioned choice. Underworld development team with AI support from Claude Code
…rapolated, never confiscated (#629) A fault specified from a structural model has its extent as data; the wrapper previously took the grid as the band and labelled the slip surface inset_rings inside it, so the requested fault came out smaller than specified. Inverted: the grid IS the fault, labelled on precisely the supplied points, and the band is built on the sheet continued margin_rings rings outward along its own end tangents (_extend_grid — linear per ring, corners consistent, normals continued the same way and renormalised; no curvature the data never asserted). The 2:1 bridge level subsamples the extended parametrisation, so nesting is unchanged. margin_rings >= 1 is enforced: a split may never reach the band rim, and the margin comes from invented surround. Measured on the curved rig (29x29 fault, margin 2): 1,566 of the 1,568 requested triangles split — only the two corner triangles erode (the splitter's no-interior-vertex refusal, a half-cell nick) — nesting 867/867, chain 95/80/95 nnz/row, velocity 8 iterations. Peak slip 0.1737 against 0.1523 under the old inset: the confiscated rings were biasing the physics by 14%, and the honoured value matches the full-patch embed reference (~0.174). GAMG parity exact (slip 0.1737, leak 2e-17, 70 its vs 8). Underworld development team with AI support from Claude Code
… parametrisation (#629) The 2-D ladder was transfinite and straight-only; the S-fault rig (San Andreas bend + through-line branch, designed with Louis) needs bent traces. _ladder_curved_assembly_2d is the 3-D extrusion one dimension down: the polyline resampled equispaced in arclength, offset ±width/2 along mitred vertex normals (constant width through turns, sharp turns refused), three rails of shared vertices, alternating diagonals, mixed-orientation triangles a refusal. No gmsh, no CAD. Straight polylines keep the gmsh transfinite path, so the recorded composed benchmark stays bit-identical (verified). The nesting contract carries over from 3-D and is enforced by API shape: coarser levels must SUBSAMPLE one fine parametrisation — the assembly accepts precomputed (samples, reach) for exactly that, and independently recomputed reach vectors were measured to break rail coincidence (only the spine nested). With the shared parametrisation: 132/132 mid band vertices coincide on the rig's tanh S trace. Underworld development team with AI support from Claude Code
…#629) The 2-D production path with the full contract set: each supplied polyline IS a fault, its points becoming the band's spine vertices verbatim (the curved ladder's (samples, reach) patches — one parametrisation per trace, extended margin_rings by tangent continuation); traces are placed sequentially as stop-short strands (the junction ruling) and cut+split in ONE add_fault network call — chained calls were measured to drop the earlier fault's pairing records. The (samples, reach) tuple patch is wired through the 2-D ladder hook with the domain gate shared between paths. visualisation/glyphs.py brought over from feature/stress-glyphs (PR #601) for the rig's standard figures: dCFF slip-vs-welded and principal-stress trajectory nets over a fine grey mesh. Validated on the S-fault rig (tanh S + through-line branch, the geometry designed with Louis): 1,858 cells, native chain (24-27 nnz/row), sub-second warm solves, machine-zero leak on both strands, first partitioning number branch/main = 0.41, locked welded state reproducing uniform shear to 2.6e-5. Tests: curved-ladder nesting + refusals, and the two-strand wrapper with honoured trace vertices. Underworld development team with AI support from Claude Code
… — render nodally Louis's catch on the S-fault rig's dCFF panel: resampling a recovered P1 field onto a regular pixel grid via uw.function.evaluate produces artefacts across the elements — grid points that straddle a facet get located into a neighbouring cell with slightly-off reference coordinates. The rule added to the skill: render derived fields NODALLY on the mesh's own triangulation (vertex evaluation is exact for P1 whichever cell the locator picks; VTK interpolates within elements), and on a split mesh never Delaunay the DOF cloud — it re-triangulates across the slit. Underworld development team with AI support from Claude Code
…outcrop-3d-cap # Conflicts: # src/underworld3/cython/petsc_generic_snes_solvers.pyx # src/underworld3/discretisation/discretisation_mesh.py # tests/test_0203_solver_wallclock_guard.py
… trace labels are refused The fault-footprint mask is now returned by both wrappers instead of living only in documentation: place_fault_ribbon reports info["footprint"] and place_fault_ribbon_2d reports per-label info["footprints"] — band cells whose nearest extended-parametrisation sample is a USER point, so painted rheology and fac_zone keys can no longer silently extend into the extrapolated tip margin (the mistake measured twice in the campaign: whole-band paint put tip lobes about two elements past the mapped tips). The test is geometry-free — the parametrisation itself carries the user/extension distinction — so it works for any curved strand. place_fault_ribbon_2d also refuses duplicate trace labels at the API boundary rather than failing downstream with an ambiguous-boundary shape inside the add_fault network call. Both fixes carry assertions in test_0855 (footprints strictly inside the band, subset of it, per strand; the duplicate-label refusal). Underworld development team with AI support from Claude Code
…st-local-h headroom in test_0861 test_1021 asserted the pre-#629 blanket rule (the rotated coarse solve is always svd). The campaign gated it: svd only when the rotated problem carries a VERIFIED rigid-rotation null mode, because blanket svd was measured as most of ~0.8 s per V-cycle (#622). The bundle fixtures pin the inner annulus boundary, so no rotation mode survives and redundant/LU is the correct coarse solve there — the assertions now say so. The svd arm keeps its regression protection through a new test: rotated free-slip on BOTH boundaries leaves the genuine rotation mode, the builder must verify and record it, and the coarse solve must be svd (this is exactly where redundant/LU hits its zero pivot, #306). test_0861's two setback placements failed only on CI: with the 0.6 default clearance the carve cavity extends ~(clearance+1)*h_local, and at the worst local h a different gmsh version deals this box that overruns the 0.25 setback margin. clearance=0.3 (the measured production thin-shell choice) keeps the cavity inside the margin at worst-case h; the tests pass locally under both settings. test_0054's failure is the CI-flaky hang-watchdog family (development's own latest CI failed sibling test_0053); no change here — rerun. Underworld development team with AI support from Claude Code
…ver dumps sources (#638 CI, test_0054) Root cause, established with native thread samples of two live hangs: faulthandler.dump_traceback_later(repeat=True) walks live frames from its C thread without synchronisation, and fired against a still-importing interpreter that walk loops forever (locally: the C thread pinned in dump_traceback for the whole sample window, the main thread starved mid-import) or reads garbage and dies at SIGSEGV (the CI -11). The env-armed watchdog was arming during `import underworld3.mpi` — inside the very import it then dumped over. The failure is conditional, which made it look flaky: it needs the import slow (cold bytecode caches, as on CI or straight after a build) and piped child output (as the test runs), and this branch's larger import graph pushed the import past CI's 1.0 s trigger where development stays under it. Fixes, each a real hardening: (1) mpi.py preloads traceback/linecache/tokenize so the reporter thread never enters the import system; (2) _stack_dump formats with lookup_lines=False — file names, line numbers and functions carry the hang report, and the source-text reads were reporter-side IO against modules mid-import; (3) report() re-arms its own Timer only — re-issuing dump_traceback_later cancels the C thread and cond-waits on it mid-dump (the sampled deadlock triangle); (4) the structural fix: the environment-armed watchdog arms at the END of `import underworld3`, never during it, trading self-coverage of the import graph for safety everywhere the tool exists for (documented in _watch_from_environment). Validation: the previously first-run-reproducing context (piped children after a fresh build, 0.2 s interval) went from 12/15 frozen to clean in the test's own scenario; the actual test passes 6/6 locally. Known latent issue, pre-existing and out of scope: a natural process exit while a repeat dump is in flight can hang finalisation (the test SIGKILLs and never sees it). Underworld development team with AI support from Claude Code
…pines A whole network of 2-D polylines in ONE call: the ribbons are fused in CAD (touching strands, junctions free) and every spine's interior points and segments are embedded in the fused face before gmsh meshes it, so the split cut walks exact vertices at any resolution (#595: nothing snaps). This unifies what the sequential ladder (robust cuts, no touching) and the plain fuse (touching, routed cuts that failed at fine resolution and for open-gap stepovers) each did only half of. Measured on the S-fault rig (kissing Y at gap w/2, shared-band stepover, branch): coarse and fine split cuts leak ~1e-17 on all four strands; the open-gap and fine cases that failed on the plain fuse now pass. Underworld development team with AI support from Claude Code
…ing pair A main line and a splay leaving its side at half the band width, placed with mesher='network': interior spine points are mesh vertices to 1e-9, area is conserved, angles stay above 10 degrees, and one add_fault network call cuts both strands. The plain fuse is the negative control (same pair, no spine vertices). Underworld development team with AI support from Claude Code
A fault was specified once but realised in two unrelated ways: the network object hardwired split-node (build -> add_fault, apply_contact -> add_fault_bc, no width at all), while a volumetric weak plane had to be assembled by hand out of place_thin_volume and painted fields. The two were then not comparable, because they were not even the same mesh. build(width=..., realisation="split"|"ti") places ONE ribbon band along every prepared piece and either cuts it or leaves it whole, so the cells are identical either way (test_0858 asserts it, with the split's duplicated vertices as the only difference). apply() imposes whichever was built and refuses the other; ti_fields paints the weak-plane viscosity on the honoured footprints and the director from the nearest SEGMENT of the owning strand, so a curved trace carries its own orientation and re-sampling it does not move the answer. slips() reports each realisation in its own quantity - the pair jump, or the tangential velocity jump across the layer, gauged one half-width plus a cell out. What width MEANS is the real asymmetry and is documented as such: a resolution parameter for the split (the band exists to give the cut its vertices), constitutive for the weak plane (V = 2 e_nt w), and at a TI width there may be no room left to cut. Supporting changes: place_fault_ribbon_2d gains mesher="network" (the whole set placed in one fused call, so strands may touch - promoted from the S-fault rig's place_rig) and returns the band mask, the extended spines and the width; the network object retains a Surface per piece, so friction, accumulated slip and damage live on the fault rather than in whichever realisation is current; build's mesher= now takes the choices its own dimension offers instead of a separate band_mesher. width=None keeps the previous no-band split path unchanged. Underworld development team with AI support from Claude Code
Carried over an untested assumption in the last commit: that a TI-scale band might leave no room to split. Measured, and it is wrong. The network mesher embeds the spine's points AND segments in the fused band, so the cut chain is part of the mesh by construction; the split is complete, with exact vertex coincidence, from w = h_far down to w = h_far/10, straight and curved traces alike, at two and three elements across the band. Every interior node duplicates and the two tips stay welded - the tip rule, not a shortfall. The only refusals in the sweep were at w = 2 h_far, where the tip margin carried the carve cavity into the domain wall. That is a band too large for its position, and it gets harder as the band widens, not as it thins. So the realisation is a free choice rather than something the mesh grants or refuses, and the asymmetry between split and TI is only what the width MEANS - a resolution parameter against a constitutive length. Docstrings, the fault-networks guide and a regression test at two widths now say that. Sweep and table: ~/+Simulations/fault_split_at_ti_width. Underworld development team with AI support from Claude Code
The last two commits exposed the band as the TI realisation's carrier and as nothing else, which reads it backwards. A segmented fault does its interesting work at the strand tips and in the ligaments where one cut stops short of the next, and damage in those places needs cells to live in - so the SPLIT wants the surrounding embedding as much as the weak plane does. net.band and net.footprints expose the masks in either realisation, and net.band_yield(tau_y) gives the rheology that goes with them: von Mises confined to the band, everything outside it far too strong to yield (promoted from the S-fault rig). A released flank sits far below tau_y and is untouched, while tips, welds and junction slivers sit far above and yield by themselves - the breakdown lands where the mechanics puts it rather than where a geometric plug was placed. damage_yield stays the tool for studying the junction glue itself. The guide also now records the correspondence the coordination contract agreed: eta_f = eta_band / w, so the band's V = 2 e_nt w is what the contact replaces with a genuine slip rate. Underworld development team with AI support from Claude Code
… tangent, not asymmetry For any eta(edot_II) law the consistent tangent is symmetric (the rank-one term eta' edot x edot / edot_II); what stalls Newton with a large yielded fraction is the hard-Min plastic branch's zero tangent stiffness along the flow (2 eta + 2 eta' edot_II = 0). Maintainer review, 2026-08-26. Underworld development team with AI support from Claude Code
A fresh Mesh built from a surgery DM owned no geometric-MG tail, and Mesh.add_fault returned a standalone child, so every solver on a placed fault mesh that did not call set_custom_fmg by hand silently fell to GAMG - the Chebyshev-smoothed configuration a nonlinear solve should never be handed by default. custom_mg.adopt_hierarchy(mesh, base, fac_zone) makes the placed mesh own the base's coarse tail (a base that is itself an adapt child contributes its own tail plus itself) with the band as a MESH-OWNED FAC zone, which build_transfers now honours; the 2-D ribbon wrapper adopts it after placement, and add_fault's child inherits the tail (a cut is the same grid re-represented) without the zone (a split fault needs no patch). Measured on the S-fault rig: plain stokes.solve() on both realisations drives the 3-level custom-P hierarchy, rotated path included, with no fallbacks. Regression test in test_0857. Underworld development team with AI support from Claude Code
The annulus between a placed assembly's skin and its carve ring was meshed at the skin's own segmentation throughout: gmsh extended the constrained curves' sizes inward and took the minimum, so a network of ribbons paid for a rung-resolution collar around every strand (the #629 "fill shell was the fat" finding recurring on the network path; measured on the S-fault rig: 1662 collar cells against 1748 band cells). The fill now passes a size callback interpolating from the skin size to the ring's median edge length by relative distance, and _gmsh_fill_2d makes that callback authoritative (extend-from-boundary, from-points and from-curvature off). Measured: collar 1662 -> 1292 cells, total 3876 -> 3506; the residual collar is the transition a 6x size jump from band to a uniform base needs - a graded base (adapt-on-top toward the traces) is the cure for the rest. Underworld development team with AI support from Claude Code
`global_evaluate` deadlocked whenever a query set left some rank with no
points. At np=4 the migration test hung indefinitely -- 900 s with no progress
on a 300-point query -- while passing at np=2.
Native stacks during the hang name both halves:
3 ranks DMSwarmMigrate -> DMSwarmDataExCreate -> MPI_Comm_dup
1 rank DMLocatePoints -> DMGetBoundingBox -> MPI_Allreduce
A mismatched collective, not slowness and not a location failure: every rank
was inside MPI, none in locator code.
The cell-location policy (`mesh._hint_is_authoritative`) decides whether the
barycentric hint may bypass PETSc's `DMLocatePoints`, which is COLLECTIVE on
the mesh DM communicator. The policy is a mesh capability, so all ranks agree
on it -- instrumenting confirmed `auth=True` on all four.
It was then discarded in transit. `create_structure` had no hint array to pass
when a rank held zero points, and passed `hintAuthoritative = 0` HARDCODED
beside the NULL:
ierr = DMInterpolationSetUp_UW(self._ipInfo, dm, 0, 1, NULL, 0)
so that rank alone took the DMLocatePoints branch. Forwarding the caller's
policy is the fix; `petsc_tools.c` additionally has to accept a NULL hint when
there are no points to hint at, since the bypass is trivially correct with
nothing to locate.
Why it hid at np=2: the test biases every point into x > 0.5, and at np=4 one
rank owns solely x < 0.5 and so receives nothing, where at np=2 the coarser
partition leaves both ranks straddling the split. Measured ownership at np=4:
rank 1 gets 0 of the 300 points, and rank 1 is the rank the watchdog roll call
singled out.
Two wrong turns are worth recording so they are not retried. A pre-touch of
`mesh.dm.getBoundingBox()` does nothing -- PETSc does not cache it, so the
later call inside the locator reduces again. And forcing the DMInterpolation
cache decision to be unanimous does nothing here either: instrumenting showed
all four ranks MISS the cache, so it was never the divergence. Both were
reverted rather than shipped.
The docstring on `_location_capability` states the assumption that made this
invisible: "deliberately NOT reduced across ranks: the evaluator runs on
COMM_SELF [...] petsc_interpolate is reached only by ranks holding points".
Both halves are false -- `DMLocatePoints(dm, ...)` uses the mesh DM's
communicator, and a trace shows all ranks entering `petsc_interpolate`,
including one with zero points. The policy does not need reducing (it already
agrees); it needs to survive the call.
Regression covers a query biased so some rank is empty, plus an unbiased
control that passed before the fix -- so a failure there means the ordinary
path broke rather than the empty-rank path being repaired. Green at np=1, 2, 4,
as is the whole of test_0760 which previously hung at np=4.
Underworld development team with AI support from Claude Code
The split representation only goes wrong at the joints, so the glue is placed rather than found. junction_cells() takes the band cells whose nearest spine point lies in a piece's extrapolated margin and which lie inside a second piece's ribbon, on nobody's cut - a kissing branch, an abutting pair, a stepover bridge - and dilates them by one vertex ring. Free tips are excluded: damage there lengthens a fault instead of joining it. junction_patch() makes those cells weak isotropic material, eta = ratio * eta_0 (default 0.01), a ratio rather than a yield stress because the joint only has to be broken and a ratio needs no stress scale; the solve stays linear. Measured on the S-fault rig at two resolutions: the patch recovers 0.8-0.97 of a continuous fault's transmission across a stepover, the slip crosses on the cut itself, the rest of the network keeps the split's answer, the weak patch reproduces a fully plastic patch to 1-2% and is insensitive to the ratio over two decades. The bare junction cells cannot repair the weld (its stiffness is the ring of intact material around the tips), which is why the ring is part of the rule. For the rule to see a joint the ribbons have to meet across it, so build() extends the tip margin at a prepared junction end until the ribbon reaches the other piece's cut, and place_fault_ribbon_2d now takes a (start, end) margin per trace. band_yield stays as a damage model for the band; the doc records why a uniform yield threshold cannot pick out the welds alone. prepare()'s near-miss double pull-back is marked as a design TODO. test_0859 checks the mask on a T plus an abutting pair (localised at the joints, nothing at the free tips, refused for the weak plane) and that the patch lands between the CUT and LONGER end members at a station on the continuation. Underworld development team with AI support from Claude Code
The near-miss pass pulled an endpoint back by a whole ligament before checking its clearance, and did so from both sides of an endpoint-to- endpoint contact, so a collinear pair 0.05 apart at h = 0.03 was left 0.17 apart. The pull is now the clearance deficit, lig - d, shared half-and-half when the two ends face each other, verified against the same target - the join is as small as the mesh allows, which is what the network's own docstring asks for. test_0859 records it; the existing prep and toolkit tests are unchanged. Underworld development team with AI support from Claude Code
Running the S-fault rig through FaultNetwork.build showed what two ribbons placed along one line do: their margins overlap with interleaved vertices and the fused placement fills the overlap with slivers (7800 band cells below 1e-6 in area at the stepover, 82 velocity iterations against 14, and a junction mask made of slivers). The rig avoided this by construction, with one spine for the main line and the cuts on its sub-chains; that construction now lives in the library. place_fault_ribbon_2d takes spines= separately from the traces to cut, and reads each trace's footprint by matching its vertices on the concatenated spine samples. build() groups pieces that continue one another (end tangents within 25 degrees, the far start within half a width of the line, within the margins' reach) onto shared spines, bridging each gap with vertices at the local rung; per-end junction margins are computed per spine. junction_cells reads "off the cut" spine by spine (a gap edge and a tip margin are spine the split did not cut) and counts the meeting of two pieces over the cut pieces continued by the default margin - a cell on the senior's flank beside a kissing tip is glue, so being on another piece's cut does not exclude it. On the rig's geometry the shipped path now reproduces the rig's own placement (4376 cells, 14 iterations, the same partition to three figures) and the library's glue releases the continuation and the kissing splay. test_0859 checks the shared spine and the absence of slivers; the placer, toolkit, prep, mesher, realisation and 3-D tests are unchanged (57 passed). Underworld development team with AI support from Claude Code
Inserting a bridge vertex into every gap claimed the stepover corridor for the gap (the honoured-paint rule paints a cell by its nearest spine vertex), so the weak plane read a closed one-rung gap as GAPPED where the rig reads it MERGED. The gap edge's two ends belong to different pieces, which the junction rule already treats as off-cut, so a gap of one rung gets no vertex; longer gaps are bridged at the local rung as before. Also records the graded-base carve failure at fine width (a TODO(BUG) at the check, and the carve_clearance remedy in build()'s docstring). Underworld development team with AI support from Claude Code
…he finest level A placed fault mesh is gathered onto its surgery rank while the multigrid tail stays load-balanced. The transfer then pairs a fine node with a coarse cell on another rank: 488 of 5614 coarse DOFs on one level of the S-fault rig had no fine image at np=2, the build fell through to the local-RBF rescue, and repairing that many columns by injection leaves a preconditioner that does not converge. adopt_hierarchy now redistributes each coarse level so every coarse cell lives on the rank holding the fine cells over it (nearest owned fine centroid, global minimum, shell partitioner) - ptest_0004's co-resident construction. The fine mesh and its FAC patch never move. On the rig at np=2 the barycentric transfer builds with serial sparsity, the solve converges in the serial 14 iterations, and the wall time equals serial (13.1 s) - correct, and no worse, as far as a fine mesh gathered on one rank allows. Also: a parallel counterpart of the serial zero-column repair for the cross-partition build, and the fallback messaging corrected - the RBF rescue has been the sparse local kd-tree interpolator since #429, not a dense transfer. fault_pair_jumps(gather=True) all-gathers the rank-local pairs for diagnostics that go on to make collective calls. ptest_0859: the network built, glued and solved at np=2 gives the serial slips and records no preconditioner fallback. Serial custom-MG suite 263 passed; ptest_0004/0845/0852/0855 pass at np=2. Underworld development team with AI support from Claude Code
…k-deadlock' into feature/fault-network-parallel
Member
Author
Adversarial reviewWe read the diff for what would break first.
Checked: serial custom-MG suite 263 passed; ptest_0004/0845/0852/0855/0859 at np=2; the S-fault rig at np=2 runs gauges and checkpoint to the serial partition. Verdict: mergeable after #656, with 2 as the follow-up to measure at np=4 in 3-D. |
…-parallel custom_mg.py: development's copy is this branch's own pre-parallel state (verified: no diff against 25c30e2), so the co-location edit on top of it is kept.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
A placed fault mesh is gathered onto its surgery rank while the multigrid tail stays load-balanced. The geometric transfer then pairs fine nodes with coarse cells on other ranks: on the S-fault rig at np=2, 488 of 5614 coarse DOFs on one level had no fine image, the build fell through to the RBF rescue, and repairing that many columns leaves a preconditioner that does not converge.
adopt_hierarchynow co-locates each coarse level with the finest (coarse cell → the rank holding the fine cells over it; shell partitioner) — ptest_0004's co-resident construction. The fine mesh and its FAC patch never move; only the small coarse levels do. Rig at np=2: barycentric transfer at serial sparsity, the serial 14 iterations, wall time equal to serial.rbf_prolongationhas been the sparse local kd-tree interpolator since custom_mg.rbf_prolongation: no conditioning path on an indefinite saddle solve, and its only test asserts convergence rather than accuracy #429. Corrected, and a parallel counterpart of the serial zero-column repair added for the cross-partition build.fault_pair_jumps(gather=True)all-gathers the rank-local pairs, for diagnostics that go on to make collective calls.ptest_0859_fault_network_parallel: the network built, glued and solved at np=2 gives the serial slips and records no preconditioner fallback.Includes the merge of #656 (the empty-rank
evaluatedeadlock, #611): with it, the rig's full diagnostic pass runs at np=2 to the serial answer. Merge #656 first; this branch then adds only the files above.Serial custom-MG suite: 263 passed. Parallel: ptest_0004, 0845, 0852, 0855, 0859 pass at np=2.
Not in scope
No rebalancing of the fine mesh (the placement stays gather-first), no GAMG path. A fine mesh gathered on one rank gives no speed-up; that is the placement policy, not this PR.
Underworld development team with AI support from Claude Code