Skip to content

diff-soup 1.2.0

Latest

Choose a tag to compare

@stmorgenstern stmorgenstern released this 11 Nov 20:26
5aad1de

Sparsity read from the equations, compiled constrained optimization in ds.optimization, and control problems in ds.optimization.control, with complex linear solves on every exact backend, a public schedule solve, and fixes to JIDA and consistent initialization.

Breaking changes

  • TrustRegion(nested_at=...) now takes the joint fit's step and, after an accepted step, polishes each member's own values with the outer values held, on lane passes that reuse the last Jacobian; with inner=TrustRegion(max_steps=0) it is the joint fit. Previously its outer step was the trust-region step of the Schur complement into the shared block, each member's response bounded by its own radius, and the polish ran before the outer acceptance. Its stop rules and verdict are now the joint fit's, and a lane that solves at an accepted trial but fails in the blocks evaluation that ends the members' polish raises ValueError naming the lane and ends the fit, as the joint fit does at an accepted point; previously that trial was rejected.
  • Removed diff_soup.problem.implicit.get_or_build_solver_view. Use ds.solver_view, which returns the same cached view.
  • Removed ds.inference.constrained_optsolve and ds.inference.InequalityConstraints. Declare the constraints as ds.optimization.Constraints on an InferenceProblem and solve it with ds.optimization.InteriorPoint or ds.optimization.AugmentedLagrangian.
  • JIDA with algebraic rows out of the error test, its stock setting, now takes a second Newton iteration when a step's first correction exceeds weighted norm 1. Previously a stale convergence rate could accept it, leaving an algebraic state far off its constraint and its gradients wrong. Results on DAEs change at the level of the solver tolerance.
  • Consistent initialization now accepts a final residual within ic_atol + ic_rtol * S, with S = ‖|F_y| |y| + |F_ydot| |ydot|‖ the size of the residual's terms at the final iterate, plus the rounding bound of its longest row, after Newton has tried the absolute target. Previously a start at the rounding floor of large states failed with INCONSISTENT_IC. ToleranceSpec(ic_rtol=0.0) keeps the absolute gate, as does ic_atol= without ic_rtol=, and so does an engine built with ic_tol= and no ic_rtol=, which previously took the step's rtol.
  • While a crossing is located, Event conditions now receive the solve's theta as args, as they do when the event is detected, so a condition can read and differentiate a parameter. Previously they received None there, and a condition that tests args is None now takes the other branch.
  • spec.build() now declares only the Jacobian entries a tangent can reach, unless an equation uses q.dt (exact_sparsity=None, the new default). Pass exact_sparsity=False for the declared blocks with a dense border per global.
  • A SolveContext now belongs to the thread that first factors in it, whatever the backend: factor and empty_factor in it from another thread, or a trace there of a function that calls them, raise FactorContractError. Create one context per thread.
  • TrustRegion on a lowered study now compiles a trial_predicate and a custom regularization into each attempt. One that reads x on the host now raises. Previously both ran in Python between the attempt's compiled calls.

Features

  • Added ds.optimization.InteriorPoint, SQP and AugmentedLagrangian, compiled solvers for problems with bounds and with equality and inequality rows declared as ds.optimization.Constraints on the problem. They run under jit and vmap, in float64 or float32, where a tolerance left unset is max(default, 100 * eps) for the iterate's dtype, and end with a TerminationStatus, among them the new LINE_SEARCH_FAILED for a line search that found no acceptable step. A variable whose lower and upper bounds are equal is fixed at its value, as Ipopt's fixed_variable_treatment="make_parameter" does. See Constrained solvers.
  • Added ds.optimization.lanes, which solves a batch of problems that differ in args and x0, split over the devices of a Layout. Its max_steps=, tol= and warm_start=, like the compiled solvers' run(max_steps=, tol=), are runtime values, one per lane or one for all, so a new value does not compile the loop again.
  • The compiled solvers differentiate their solution by the implicit function theorem, in args and in the problem's bounds: an active bound moves the coordinate it holds, and its multiplier is the objective's rate in it. A tangent on the start point, the warm start or the problem's scales raises ConfigurationError. With hessian="gauss-newton" on a least-squares problem the rule is the Gauss-Newton form, first order throughout (J^T J for the Lagrangian Hessian, no constraint curvature), which equals the exact rule at a zero-residual solution; hessian="exact" or "auto" takes the exact rule.
  • Added ds.optimization.control. A ControlProblem poses optimal control of a model's operational inputs over a horizon, with a cost, path rows and terminal rows on the solution, and lowers to an OptimizationProblem that optsolve and the compiled solvers take. receding_horizon runs the closed loop as one compiled program, and HostLoop steps it from Python for a plant outside JAX; each horizon starts from an estimator's estimate (FullState, ExtendedKalmanFilter, MovingHorizonEstimator).
  • ds.linear.{KLU,SuperLU,CuDSS} now factor and solve complex matrices (complex128, and complex64 on CuDSS) on the real symbolic analysis, solve a complex right-hand side on a real factor, and differentiate both ways.
  • Added ds.solve_schedule and ds.schedule_arrays, which run a Schedule against a model's operating table, with max_steps per segment and first and second derivatives with respect to segment durations. See Events and protocols.
  • Added ds.optimization.control.SignalControls, which drives a ControlProblem with the time signals of ds.optimization.inference.approx, such as a current on the new clamped cubic BSpline or a current then a voltage, and runs each signal on the schedule exactly. With free_durations=True each signal's duration is also a decision variable, and ControlProblem(save_fractions=) samples within every segment.
  • Added a "resistance" row to spec.control.operational and Segment(resistance=), which hold voltage = target * current.
  • Added spec.build(exact_sparsity=True), which declares only the Jacobian entries a tangent can reach.
  • Added ds.solver_view(problem), the cached view the solvers take of a problem (ds.SolverView), and ds.jacobian_nnz(problem), the entry count of its iteration matrix. See Problem interface.
  • Under jax.default_device, linear="auto" and consistent initialization choose their backend by that device's platform, and native factors are kept per device, so one process can run lanes on a GPU and on CPUs. checkpointed_map(device_groups=) accepts groups of another platform than the default backend's.
  • KLU.native_stats reports live_lu_entries.
  • SuperLU.native_stats reports singular, the factor requests that ended singular, as KLU.native_stats does.
  • Added ds.jacobian_pattern(fn, y, *args, within=None), the entries of a Jacobian that can be nonzero, read from the function's tangent map without evaluating it. Rules cover loops, branches, gathers, scatters at known indices, cumulative sums, convolutions, linear solves, jax.checkpoint, and jax.custom_vjp functions, read through their bwd; an operation without one falls back to every entry with a warning, bounded by within. See Problem interface.
  • ds.topology_report adds pullback_count, and its color_count is now the solver's own: the colors of the rows the solver colors, with the rows it holds out recovered by pullback. Previously it colored the whole pattern.
  • TerminationStatus members are now also their integer codes, so a compiled solver's SolveRecord.status compares with them under jit and TerminationStatus(code) reads a code back. Their string values are unchanged.
  • Added hessian_composition= to OptimizationProblem.from_value and LeastSquaresProblem.from_residual. Under "forward-over-forward" the gradient, the Hessians and the Lagrangian Hessians with_constraints adds are all forward mode, so InteriorPoint, SQP, AugmentedLagrangian and their implicit derivative take an objective whose inner solve supports forward mode only, such as a DAE solve under ForwardAdjoint. The default, "forward-over-reverse", is unchanged.
  • Added CheckpointAdjoint(reverse_over_reverse=True), under which a gradient through a solve can itself be differentiated in reverse mode, as in jax.grad of jax.grad, and the solve also takes jax.jvp. Forward mode over the gradient then costs about twice as much, so the default, False, keeps the previous rule.
  • Added adjoint= to InteriorPoint, SQP and AugmentedLagrangian. ds.nonlinear.ImplicitAdjoint() differentiates the solution in args by one reverse rule, a transposed KKT solve, so jax.grad costs one solve however large args is and runs through closures that differentiate in reverse mode only. The default, ds.ForwardAdjoint(), keeps the forward rule.
  • Added ds.optimization.inference.InequalityRows, rows c(own, shared, data) <= 0 on the fitted values of a lowered study, per member of a level or on the shared values, which TrustRegion(inequalities=...) keeps. Each step finds its active set by a primal active-set method on the projection that keeps the equality rows, holding a row the step would cross at its boundary and releasing a held row whose multiplier is negative, and a trial that crosses a row is moved back to it. bootstrap refuses a solver with inequalities=, and with its default solver a fit made under them, which records its row count in info["inequality_rows"]. See Optimization solvers.
  • Added the "frozen" mode of a lowered study: after prepare(modes=(..., "frozen")) each blocks evaluation keeps the Jacobians of its lanes that solved, and a "frozen" evaluation returns each lane's cost from a plain solve and its gradient on the kept Jacobian, at the cost of a value evaluation. The nested TrustRegion's polish runs on it; on a CPU the nested fit cuts its blocks and frozen calls at the members' boundaries, so a polish request runs only the members still in it.
  • TrustRegion(constraints=...) and TrustRegion(gauges=...) now also take one LinearConstraint or one Gauge, as inequalities= takes one InequalityRows.

Bugfixes

  • Fixed MAP fits of a @ds.recipe template, which took every prior from the template's first build in the process, so a later build with another prior was fitted under the first one's. Each fit now takes each prior from its own experiments' builds (per owner for a parameter that varies by owner), a GaussianCoefficientPrior centers on its own build's coefficients, builds that differ only in a prior return two problems, and a MAP fit refuses a parameter shared by experiments whose builds declare different priors.
  • Fixed member equations, whose q.p.<name> and q.fn.<name> raised AttributeError. They now read the problem's declared parameters and functions, with their tangents, as component equations do.
  • Fixed consistent initialization of a family problem whose step backend is a MemberBlock over the full state, which raised a ConfigurationError. Initialization now solves its smaller system on the backend linear="auto" chooses for its size.
  • Fixed SaveAt(output="projected") rows that came back NaN at a tight atol. Projected samples now pass the same gate as the solve's own initialization.
  • Fixed JIDA solves that stalled at a tight atol after a short step, where Newton's corrections are pure rounding. A correction within twice the rounding floor on a fresh matrix is now accepted as the root.
  • Fixed the derivative of consistent initialization with respect to the start time.
  • Fixed TrustRegion with constraints= or gauges= from a start off the equality rows, which could return the start as converged when the step onto the rows raised the cost. The start now moves onto the rows first, by the least change of its free coordinates.
  • Fixed TrustRegion on a lowered study, whose radius collapsed after a trial of zero length. Such a trial now keeps the radius.
  • Fixed jax.grad in t of CubicHermite and PiecewiseLinear on a knot, which was half the derivative of the segment the knot selects. It is now the whole one: the knot's slope for CubicHermite, and for PiecewiseLinear the slope of the segment to its right (of the last segment on the last knot).
  • Fixed SolveContext.resident_bytes on KLU, which read uninitialized memory for any factor with a singleton block.
  • Fixed a MAP fit of a joint problem, which read every shared parameter's prior from one handle. Each occurrence now takes its prior from the declarations of the experiments behind it, and a MAP refuses an occurrence whose schema records none.
  • A KLU, SuperLU or CuDSS factor lowered into a program over several devices of one process now raises FactorContractError. Previously XLA failed with an internal error, or, in a shard_map body, the devices shared one native factor.
  • Complex values on an analysis declared spd=True, and on a CuDSS analysis with downcast_factors or emulation_mantissa_bits, now raise FactorContractError. Previously cuDSS factored them as a real symmetric positive definite matrix, or ignored the option.
  • Fixed SuperLU on more than one thread, which returned a wrong solution for a few factors in a hundred with a success status. Its factors on any number of threads now equal one thread's, bit for bit; the build patches SuperLU_MT 4.0.2 (diff_soup/_internal/superlu_ffi/patches/).
  • Fixed CuDSS solves that could apply another factor and report success: two executions of one compiled program at once on one SolveContext, or two calls of one jitted function inside one program, which shared a factor slot. Each call now keeps its own factor, and a solve always applies its own handle's factor.
  • A released Analysis of KLU, SuperLU or CuDSS now frees its native analysis, its call sites and its factors in every SolveContext, DEFAULT_CONTEXT included, once no compiled program that factors with it exists. Previously the sites kept every analysis for the life of the process.
  • Fixed a SuperLU solve with real values of a shared storage that a refactor with complex values filled earlier in the same program, which returned nan with a success status. It now factors the real values first, as KLU does.

Performance

  • A vmapped solve keeps fewer copies of its saved output (two state rows per save time, from five), and ds.topology_report no longer builds an n-by-n adjacency for a problem with a full row. An LFP half cell with 206,518 states and 1,001 saved times now completes on a 12 GB GPU.
  • JIDA does its whole-array work only on the path an attempt takes, with identical results.
  • Outside vmap, the branches of an implicit step (JIDA's history update, the Jacobian refreshes, the retries) compile to one conditional and are traced once. Under vmap a branch still runs only when some lane takes it, with one compiled copy of each branch.
  • Consistent initialization's derivative skips the start-time term when the start time is constant, and its relative gate evaluates the residual's scale only where the absolute target was missed, on the algebraic rows alone on the reduced route.
  • spec.build(exact_sparsity=True) reads its pattern with one compiled program instead of one per operation, 0.85 s instead of 3.6 s at 18 families, and neither build colors the declared pattern unless physics_jacobian_blocks is called. Loops, cumulative sums, convolutions and linear solves in a residual keep their pattern instead of filling it.
  • Member rows that read every site of their member are recovered by shared pullbacks instead of one color per site: an LFP half cell with 50 volumes of 10 particles needs 8 colors instead of 108.
  • TrustRegion on a lowered study now runs each attempt as two compiled programs, one that proposes the trial and one that judges it, with the MAP priors' rows, the trial predicate and the active set of inequalities= inside them; with nested_at=, each polish iteration is two more, for all members at once. Fits of one lowered study share their compiled programs, so a second fit compiles nothing: a second joint fit of a 192-experiment study takes 0.10 s on a CPU, from 2.4 s. A trial_predicate or a custom regularization that reads x on the host now raises when it is traced.
  • Inside one compiled program, each call of a jitted function that factors on KLU or SuperLU now keeps its own factor: k calls refactor k times a run, from 2k.

Other

  • Three undocumented module paths moved: diff_soup.timestepping.consistent_ic and diff_soup.problem.consistent_init are now diff_soup.problem.implicit.consistent_init, and diff_soup.timestepping.vsvo.common.weights is now diff_soup.utils.weights.
  • ds.optimization.inference.approx.FluxNet raises AttributeError like any other missing name. It raised a TypeError naming eqx.nn.MLP since its removal.
  • CuDSS's contract_stats no longer reports perturbed_pivot_factorizations, which was always 0: every cuDSS factor runs with pivot perturbation off.
  • The cuDSS extension's default CUDA architectures add 87-real (Jetson Orin).
  • The wheels now include SuperLU_MT, so linear="superlu" needs no source build.
  • Added wheels for Linux on ARM (aarch64).
  • A diff-soup environment variable set to an empty string now counts as unset and takes its default. Previously an empty DIFFSOUP_PROCESSES or DIFFSOUP_NODES raised ValueError.
  • diff_soup.inference is now diff_soup.optimization.inference, and optsolve, OptResult, the optimization problems and the solvers are also at the top of ds.optimization. ds.inference and every diff_soup.inference import path still work for this release, return the same objects and warn once with a DeprecationWarning; they will be removed in 1.3.

Full Changelog: v1.1.0...v1.2.0

Wheels for CPython 3.12 and 3.13 on Linux x86_64 and aarch64 (manylinux_2_28). They carry the KLU and SuperLU_MT sparse direct solvers. Install the cuDSS solver with pip install "diff-soup[cu12]" or "diff-soup[cu13]". SHA256SUMS lists the checksums.

Documentation: https://stmorgenstern.github.io/diff-soup/

Full Changelog: v1.1.0...v1.2.0