Skip to content

A warning is not observability: 10 of 12 preconditioner fallbacks leave no readable state #484

Description

@lmoresi

Five defects of the same shape have been found independently over the last few weeks — #276/#478, #467, #468, #477, #425. Every one of them degrades to something that succeeds: GAMG converges, richardson converges, the solve returns a correct answer. So none presented as a failure. They presented months later as "this is a bit slow", or as a parameter sweep returning identical rows, which reads as "this variable does not matter" rather than "this variable was never set". That cost a twelve-cell sweep on #477 and a wrong premise in #471.

This issue is about the property that makes them all invisible, rather than any one of them.

Inventory of the preconditioner-configuration fallbacks

site substitutes observable?
_apply_preconditioner_options single-field gate geometric MG → GAMG warns only on explicit fmg
_apply_preconditioner_options n_levels <= 1 geometric MG → GAMG warns only on explicit fmg
_apply_preconditioner_options override latch user's bundle → managed bundle nothing (fixed in #471)
auto_inject_custom_mg barycentric→RBF retry sparse transfer → dense warns
auto_inject_custom_mg build failure geometric MG → default PC warns
auto_inject_custom_mg dimensional guard skips the auto-pickup warns
_assert_finest_spans_operator swallow skips a correctness guard nothing
CustomMGHierarchy.build setUp swallow proceeds on a possibly-stale DM section nothing
_install_velocity_block_transfers PETSc-error path alternative assembly route nothing (same outcome)
_enforce_galerkin_for_geometric_mg forces pc_mg_galerkin=both warns
rotated_bc no hierarchy geometric MG → GAMG velocity_pc
rotated_bc no pressure mass 1/μ-mass Schur → selfp + jacobi schur_pre

Only the last two leave readable state. Both were added in #465 — and they are the only reason #467 was findable at all.

Three things the current state cannot do

1. A warning is not observability. Warnings are filtered, deduplicated by Python's warning registry (so a fallback that fires every timestep reports once), and invisible in a long log. Readable state can be asserted on in a test. There is currently no public way to ask a solver which preconditioner did I actually get_pc_managed_value and _pc_user_override are private.

2. "Couldn't" and "chose not to" are indistinguishable. GAMG-because-no-hierarchy and GAMG-because-we-do-not-trust-DMCreateInjection (#478) read identically, and only one of them is a bug. A user cannot tell whether to go and build a hierarchy or to file an issue.

3. Two silent swallows disable correctness guards. _assert_finest_spans_operator returns early when it cannot read the operator — its own docstring calls it a guarantee. CustomMGHierarchy.build swallows a failed snes.setUp() and proceeds with whatever DM section is current, which is exactly the case (a stale section on an adapt() child) the guard exists for. Both are defensible as written; both chose "do not crash the solve" over "say the check did not run".

Suggested direction

Generalise the velocity_pc / schur_pre pattern: a small reported record of what the managed block actually resolved to and why — the preconditioner that was installed, whether it was the requested one, and if not, which of unavailable / declined / build failed / user override applies. Readable on any solver, not just the rotated path, and assertable in a test.

Related, and cheap alongside it: the barycentric→RBF retry silently switches to a dense transfer whose Galerkin coarse operators are dense too. The code comment is explicit that this "rescues correctness but does not scale" and should be treated as a performance cliff — but a long run cannot tell you it happened, and no test can assert which builder ran.

This is a design change touching every managed solver, so it wants its own PR rather than riding on a fix. Filing it because the individual instances keep being found one at a time, and the reason they are hard to find is this.

Found by the solver configuration audit, docs/reviews/2026-07/solver-configuration-reachability-audit.md (branch docs/solver-config-audit), findings F-6, F-7 and F-8.

Underworld development team with AI support from Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions