Skip to content

Releases

diff-soup 1.2.0

Choose a tag to compare

@stmorgenstern stmorgenstern released this 11 Nov 20:26
5aad1de

Sparsity read from the equations, compiled constrained optimization in ds.optimization, and control problems in ds.optimization.control, with complex linear solves on every exact backend, a public schedule solve, and fixes to JIDA and consistent initialization.

Breaking changes

  • TrustRegion(nested_at=...) now takes the joint fit's step and, after an accepted step, polishes each member's own values with the outer values held, on lane passes that reuse the last Jacobian; with inner=TrustRegion(max_steps=0) it is the joint fit. Previously its outer step was the trust-region step of the Schur complement into the shared block, each member's response bounded by its own radius, and the polish ran before the outer acceptance. Its stop rules and verdict are now the joint fit's, and a lane that solves at an accepted trial but fails in the blocks evaluation that ends the members' polish raises ValueError naming the lane and ends the fit, as the joint fit does at an accepted point; previously that trial was rejected.
  • Removed diff_soup.problem.implicit.get_or_build_solver_view. Use ds.solver_view, which returns the same cached view.
  • Removed ds.inference.constrained_optsolve and ds.inference.InequalityConstraints. Declare the constraints as ds.optimization.Constraints on an InferenceProblem and solve it with ds.optimization.InteriorPoint or ds.optimization.AugmentedLagrangian.
  • JIDA with algebraic rows out of the error test, its stock setting, now takes a second Newton iteration when a step's first correction exceeds weighted norm 1. Previously a stale convergence rate could accept it, leaving an algebraic state far off its constraint and its gradients wrong. Results on DAEs change at the level of the solver tolerance.
  • Consistent initialization now accepts a final residual within ic_atol + ic_rtol * S, with S = ‖|F_y| |y| + |F_ydot| |ydot|‖ the size of the residual's terms at the final iterate, plus the rounding bound of its longest row, after Newton has tried the absolute target. Previously a start at the rounding floor of large states failed with INCONSISTENT_IC. ToleranceSpec(ic_rtol=0.0) keeps the absolute gate, as does ic_atol= without ic_rtol=, and so does an engine built with ic_tol= and no ic_rtol=, which previously took the step's rtol.
  • While a crossing is located, Event conditions now receive the solve's theta as args, as they do when the event is detected, so a condition can read and differentiate a parameter. Previously they received None there, and a condition that tests args is None now takes the other branch.
  • spec.build() now declares only the Jacobian entries a tangent can reach, unless an equation uses q.dt (exact_sparsity=None, the new default). Pass exact_sparsity=False for the declared blocks with a dense border per global.
  • A SolveContext now belongs to the thread that first factors in it, whatever the backend: factor and empty_factor in it from another thread, or a trace there of a function that calls them, raise FactorContractError. Create one context per thread.
  • TrustRegion on a lowered study now compiles a trial_predicate and a custom regularization into each attempt. One that reads x on the host now raises. Previously both ran in Python between the attempt's compiled calls.

Features

  • Added ds.optimization.InteriorPoint, SQP and AugmentedLagrangian, compiled solvers for problems with bounds and with equality and inequality rows declared as ds.optimization.Constraints on the problem. They run under jit and vmap, in float64 or float32, where a tolerance left unset is max(default, 100 * eps) for the iterate's dtype, and end with a TerminationStatus, among them the new LINE_SEARCH_FAILED for a line search that found no acceptable step. A variable whose lower and upper bounds are equal is fixed at its value, as Ipopt's fixed_variable_treatment="make_parameter" does. See Constrained solvers.
  • Added ds.optimization.lanes, which solves a batch of problems that differ in args and x0, split over the devices of a Layout. Its max_steps=, tol= and warm_start=, like the compiled solvers' run(max_steps=, tol=), are runtime values, one per lane or one for all, so a new value does not compile the loop again.
  • The compiled solvers differentiate their solution by the implicit function theorem, in args and in the problem's bounds: an active bound moves the coordinate it holds, and its multiplier is the objective's rate in it. A tangent on the start point, the warm start or the problem's scales raises ConfigurationError. With hessian="gauss-newton" on a least-squares problem the rule is the Gauss-Newton form, first order throughout (J^T J for the Lagrangian Hessian, no constraint curvature), which equals the exact rule at a zero-residual solution; hessian="exact" or "auto" takes the exact rule.
  • Added ds.optimization.control. A ControlProblem poses optimal control of a model's operational inputs over a horizon, with a cost, path rows and terminal rows on the solution, and lowers to an OptimizationProblem that optsolve and the compiled solvers take. receding_horizon runs the closed loop as one compiled program, and HostLoop steps it from Python for a plant outside JAX; each horizon starts from an estimator's estimate (FullState, ExtendedKalmanFilter, MovingHorizonEstimator).
  • ds.linear.{KLU,SuperLU,CuDSS} now factor and solve complex matrices (complex128, and complex64 on CuDSS) on the real symbolic analysis, solve a complex right-hand side on a real factor, and differentiate both ways.
  • Added ds.solve_schedule and ds.schedule_arrays, which run a Schedule against a model's operating table, with max_steps per segment and first and second derivatives with respect to segment durations. See Events and protocols.
  • Added ds.optimization.control.SignalControls, which drives a ControlProblem with the time signals of ds.optimization.inference.approx, such as a current on the new clamped cubic BSpline or a current then a voltage, and runs each signal on the schedule exactly. With free_durations=True each signal's duration is also a decision variable, and ControlProblem(save_fractions=) samples within every segment.
  • Added a "resistance" row to spec.control.operational and Segment(resistance=), which hold voltage = target * current.
  • Added spec.build(exact_sparsity=True), which declares only the Jacobian entries a tangent can reach.
  • Added ds.solver_view(problem), the cached view the solvers take of a problem (ds.SolverView), and ds.jacobian_nnz(problem), the entry count of its iteration matrix. See Problem interface.
  • Under jax.default_device, linear="auto" and consistent initialization choose their backend by that device's platform, and native factors are kept per device, so one process can run lanes on a GPU and on CPUs. checkpointed_map(device_groups=) accepts groups of another platform than the default backend's.
  • KLU.native_stats reports live_lu_entries.
  • SuperLU.native_stats reports singular, the factor requests that ended singular, as KLU.native_stats does.
  • Added ds.jacobian_pattern(fn, y, *args, within=None), the entries of a Jacobian that can be nonzero, read from the function's tangent map without evaluating it. Rules cover loops, branches, gathers, scatters at known indices, cumulative sums, convolutions, linear solves, jax.checkpoint, and jax.custom_vjp functions, read through their bwd; an operation without one falls back to every entry with a warning, bounded by within. See Problem interface.
  • ds.topology_report adds pullback_count, and its color_count is now the solver's own: the colors of the rows the solver colors, with the rows it holds out recovered by pullback. Previously it colored the whole pattern.
  • TerminationStatus members are now also their integer codes, so a compiled solver's SolveRecord.status compares with them under jit and TerminationStatus(code) reads a code back. Their string values are unchanged.
  • Added hessian_composition= to OptimizationProblem.from_value and LeastSquaresProblem.from_residual. Under "forward-over-forward" the gradient, the Hessians and the Lagrangian Hessians with_constraints adds are all forward mode, so InteriorPoint, SQP, AugmentedLagrangian and their implicit derivative take an objective whose inner solve supports forward mode only, such as a DAE solve under ForwardAdjoint. The default, "forward-over-reverse", is unchanged.
  • Added CheckpointAdjoint(reverse_over_reverse=True), under which a gradient through a solve can itself be differentiated in reverse mode, as in jax.grad of jax.grad, and the solve also takes jax.jvp. Forward mode over the gradient then costs about twice as much, so the default, False, keeps the previous rule.
  • Added adjoint= to InteriorPoint, SQP and AugmentedLagrangian. ds.nonlinear.ImplicitAdjoint() differentiates the solution in args by one reverse rule, a transposed KKT solve, so jax.grad costs one solve however large args is and runs through closures that differentiate in reverse mode only. The default, ds.ForwardAdjoint(), keeps the forward rule.
  • Added ds.optimization.inference.InequalityRows, rows c(own, shared, data) <= 0 on the fitted values of a lowered study, per member of a level or on the shared values, which TrustRegion(inequalities=...) keeps. Each step finds its active set by a primal active-set method on the projection that keeps the equality rows, holding a row the step would cross a...
Read more

diff-soup 1.1.0

Choose a tag to compare

@stmorgenstern stmorgenstern released this 29 Oct 17:03

New tutorials and worked examples, and the fixes and features found and added while writing them.

Breaking changes

  • spec.observe.{spatial_mean,boundary_mean} now average. fn returns one value per node, weighted by cell volume or facet measure. Previously the name was only a label. A scalar-valued fn is now refused.

    # Before (1.0.0): fn computed the mean
    spec.observe.spatial_mean(name="T_avg", fn=lambda q: jnp.mean(q.T))
    # After: fn returns nodal values
    spec.observe.spatial_mean(name="T_avg", fn=lambda q: q.T)
  • spec.bc.neumann now imposes du/dn = s on FVM, FDM and FEM. Previously it was an alias of spec.bc.flux, which imposes D du/dn = s. name= and cross_facet= are now on flux only.

    # Before (1.0.0): neumann was an alias of flux
    spec.bc.neumann("u", on="left", value=s)      # imposed D du/dn = s
    # After: say which one you mean
    spec.bc.flux("u", on="left", value=s)         # D du/dn = s
    spec.bc.neumann("u", on="left", value=s)      # du/dn = s
  • Grid boundary conditions and boundary integrals in 2D and 3D now weight each selected cell by one face. Previously edge and corner cells were counted more than once. A selection spanning several faces now raises. Pick one with axis= and side=.

  • Fixed the sign of flux and Robin terms on clockwise meshes, which includes every mask_to_mesh mesh. Element volumes now use |det J|. A tangled element now raises.

  • FEM operators now read a 1D coefficient by its length: num_dofs values are nodal and n_cells values are per cell. Previously a per-cell array was read as nodal.

  • The residual form q.dt(...) now works on meshes (previously its coefficients were dropped), and keeps the boundary conditions of the field it writes on grids and meshes alike.

  • NewtonTrustRegion and LevenbergMarquardt now end STALLED (converged=False) when the trust radius or damping holds a step short of a stopping test. Previously these fits reported converged. result.info["stall_reason"] says why.

  • NewtonTrustRegion(hessian=) now defaults to "auto", which picks Gauss-Newton on any problem with residual rows, including every InferenceProblem. optsolve and constrained_optsolve follow. Previously the default was the true Hessian. Pass hessian="true" to get it back.

  • NewtonTrustRegion(hessian_tile=) now defaults to "auto", and a true Hessian or Gauss-Newton Jacobian too large for the device stops the fit before its first step with a ConfigurationError. Previously the default was "full".

  • sample(kernel="nuts") now adapts all chains together (adapt="pooled", warmup=200), and each chain keeps its adapted state. Previously one chain adapted for 500 steps and the others started from a perturbed fit. chains=1, adapt="window", warmup=500 reproduces the old draws.

  • Identifiability checks now skip network columns, including on multi-experiment problems, so a network-only fit no longer warns. theta_singular_values, jacobian_cond and param_group_cond now cover the non-network columns only.

  • stats.linear_solve_mode now reports "fp32_factor" for Precision.fp32_factor(). Previously it reported "fp32", which now means end-to-end float32 only.

Features

  • Added ds.compile, which compiles a pdesolve call without running it. The result runs on new t_span, y0 and theta of the same shapes and dtypes, and reports compile_seconds, cost_analysis() and memory_analysis(). See Running a simulation.

    compiled = ds.compile(problem, solver, (0.0, 0.1), y0)
    sol = compiled((0.0, 0.2), y0)
  • theta[handle] reads a parameter, and theta.replace(handle, value) writes every copy of it. A theta that went through jax.tree.map, a device transfer or a tracer still works with ds.default_theta. See Learnable parameters.

  • Added ds.parameter_gradient(grads, handle), which reads dloss/dparam from jax.grad(loss)(theta) and sums the copies of a parameter that several equations read.

  • Added est.values(x)[handle], which reads parameters from a vector in the fit coordinate. ParameterSchema is exported from ds.inference.

  • Added recipe.spec(**inputs), which returns the ProblemSpec a recipe authors, and grid.{coordinates,meshgrid()} on both grid classes.

  • Grid transport: q.ops.advect_upwind takes velocity fields, q.ops.div_advect is the conservative form, with flux-limited face values by default (scheme="tvd", limiter="van_leer", "koren" or "minmod": second order where smooth, no new extrema at any cell Péclet number), third-order "weno3", "central" or first-order "upwind", and q.ops.div_flux takes a per-axis (Dx, Dy, Dz). spec.bc.{dirichlet,flux,robin,neumann} take axis= and side=, and the new spec.bc.inflow gives the Danckwerts condition with div_advect.

  • spec.bc.periodic(var, axes=(0,)) makes a field periodic on some axes and allows other conditions on the rest.

  • Mesh regions: TaggedMesh(cell_tags=, cell_tag_names=), q.ops.cell_indicator(name), polygons_to_tagged_mesh({name: polygon}, ...) with conforming interfaces and per-region edge tags (segment_tags=, segment_names=), and read_mesh physical groups as cell and boundary tags.

  • Mesh boundary conditions: flux, Robin and Neumann conditions, surface_measure and q.ops.boundary_integral on QUAD4, TRI6, QUAD8, HEX8, TET4, TET10, HEX20 and HEX27. A condition by node_indices selects the facets whose vertices all lie in the set.

  • On a mesh, Field composes with weak terms and nodal arrays and supports **, abs and comparisons. c * q.dt(field) takes one value of c per cell.

  • Added ds.inference.approx.WIRE, a coordinate network with the Gabor-wavelet activation of Saragadam et al. (2023) and the interface of SIREN. See Function approximation.

  • The Gauss-Newton Jacobian is now tiled: NewtonTrustRegion(hessian_tile=) applies to it, and LevenbergMarquardt(jacobian_tile=) is new. Previously an integer hessian_tile was ignored under Gauss-Newton. See Solvers.

  • NUTS options: sample takes adapt=, mass= ("diag" or "dense"), step_size= and max_tree_depth=, and draws.info["adaptation"] records what ran. See MCMC.

  • Added ds.inference.evidence(est, fit) and laplace(est, fit).log_evidence, the Laplace evidence of a MAP fit. Its stationarity check scales with the solver tolerance and precision of the problem, it accepts float32 posteriors, and a refusal names the solver tolerance. information, laplace, identifiability and predict now take a multiple-shooting fit.

  • Measurement(observe="name") reads a spec.observe declaration.

  • A fit now seeds with_start, posterior_target, information(at=), identifiability, profile, predict and OptResult.load(problem=) whenever its parameter layout matches. A rebuilt recipe or another solver tolerance no longer refuses it, and a start can be a mapping {handle: value}.

  • verify_gradient now returns a verdict: "passed", "failed" or "inconclusive".

  • ds.default_theta takes a whole module for a function parameter.

Bugfixes

  • ds.default_theta overrides no longer leak between two problems equal in content. Fixed parameters declared from one array, and functions declared from one module, are now copied.
  • optsolve(checkpoint=) now resumes LevenbergMarquardt and NewtonTrustRegion fits from their saved steps. It refuses a directory written by a fit of other parameters.
  • sample(kernel="nuts") now audits the gradient at a jittered chain start instead of the MAP, where the audit meant nothing. It refuses only a failed audit.
  • posterior_target(gradient="reverse"), log_posterior and LogDensity.from_callable now scale their reverse/forward check with the solver rtol and run it one percent away from the start, so a float32 fit passes at its mode. jax.jit and eqx.filter_jit now compile a LogDensity.
  • Fixed ExperimentSet.log_posterior, which raised on every call.
  • Meshes: a scalar source term in the residual form now builds, a first use under jax.jit no longer leaves a tracer in the cache, and read_mesh no longer reports TET10 meshes as tangled.
  • Fixed the operator-level FluxBC: the wrong sign on the low side of an FVM grid, and a crash with a per-cell D on a 2D grid. It now also works on 2D and 3D FDM grids.
  • Importing JAX before diff-soup no longer turns off constant hoisting, which made closures over large arrays compile to large programs.

Performance

  • Fitted window starts now solve each window once per evaluation instead of once per observed frame. With 110 frames in 5 windows the integrated time drops from 1265 to 110. Fitted values are unchanged up to the solver tolerance.
  • With fitted window starts, the Gauss-Newton Jacobian now pushes each direction through all windows at once. On an 803-variable FitzHugh-Nagumo fit one Jacobian takes 7.2 s instead of 31.0 s on CPU, 0.78 s instead of 3.30 s on an RTX 4070 Ti, and 0.33 GiB of compiled memory instead of 1.26 GiB.
  • Precision.fp32_factor() now solves its float32 factors unrefined, as documented. On Gray-Scott 256x256 at rtol=1e-6 on an RTX 4070 Ti, a solve drops from 5.10 s to 3.06 s with the same steps and errors.
  • Fit diagnostics now differentiate only the non-network columns, which removes most of their cost on network fits.

Other

  • Better errors: jax.grad through a solve with the default forward-only adjoint raises a ValueError namin...
Read more

diff-soup 1.0.0

Choose a tag to compare

@stmorgenstern stmorgenstern released this 27 Oct 15:07

First public release of diff-soup, a differentiable, GPU-native JAX library for stiff PDEs and DAEs.

Wheels for CPython 3.12 and 3.13 on x86_64 Linux (manylinux_2_28). They carry the KLU sparse direct solver; the cuDSS solver is installed with pip install "diff-soup[cu12]" or "diff-soup[cu13]". SHA256SUMS lists the checksums.

Documentation: https://stmorgenstern.github.io/diff-soup/