Repository navigation
Releases
diff-soup 1.2.0
Sparsity read from the equations, compiled constrained optimization in ds.optimization, and control problems in ds.optimization.control, with complex linear solves on every exact backend, a public schedule solve, and fixes to JIDA and consistent initialization.
Breaking changes
TrustRegion(nested_at=...)now takes the joint fit's step and, after an accepted step, polishes each member's own values with the outer values held, on lane passes that reuse the last Jacobian; withinner=TrustRegion(max_steps=0)it is the joint fit. Previously its outer step was the trust-region step of the Schur complement into the shared block, each member's response bounded by its own radius, and the polish ran before the outer acceptance. Its stop rules and verdict are now the joint fit's, and a lane that solves at an accepted trial but fails in theblocksevaluation that ends the members' polish raisesValueErrornaming the lane and ends the fit, as the joint fit does at an accepted point; previously that trial was rejected.- Removed
diff_soup.problem.implicit.get_or_build_solver_view. Useds.solver_view, which returns the same cached view. - Removed
ds.inference.constrained_optsolveandds.inference.InequalityConstraints. Declare the constraints asds.optimization.Constraintson anInferenceProblemand solve it withds.optimization.InteriorPointords.optimization.AugmentedLagrangian. - JIDA with algebraic rows out of the error test, its stock setting, now takes a second Newton iteration when a step's first correction exceeds weighted norm 1. Previously a stale convergence rate could accept it, leaving an algebraic state far off its constraint and its gradients wrong. Results on DAEs change at the level of the solver tolerance.
- Consistent initialization now accepts a final residual within
ic_atol + ic_rtol * S, withS = ‖|F_y| |y| + |F_ydot| |ydot|‖the size of the residual's terms at the final iterate, plus the rounding bound of its longest row, after Newton has tried the absolute target. Previously a start at the rounding floor of large states failed withINCONSISTENT_IC.ToleranceSpec(ic_rtol=0.0)keeps the absolute gate, as doesic_atol=withoutic_rtol=, and so does an engine built withic_tol=and noic_rtol=, which previously took the step'srtol. - While a crossing is located,
Eventconditions now receive the solve'sthetaasargs, as they do when the event is detected, so a condition can read and differentiate a parameter. Previously they receivedNonethere, and a condition that testsargs is Nonenow takes the other branch. spec.build()now declares only the Jacobian entries a tangent can reach, unless an equation usesq.dt(exact_sparsity=None, the new default). Passexact_sparsity=Falsefor the declared blocks with a dense border per global.- A
SolveContextnow belongs to the thread that first factors in it, whatever the backend:factorandempty_factorin it from another thread, or a trace there of a function that calls them, raiseFactorContractError. Create one context per thread. TrustRegionon a lowered study now compiles atrial_predicateand a custom regularization into each attempt. One that readsxon the host now raises. Previously both ran in Python between the attempt's compiled calls.
Features
- Added
ds.optimization.InteriorPoint,SQPandAugmentedLagrangian, compiled solvers for problems with bounds and with equality and inequality rows declared asds.optimization.Constraintson the problem. They run underjitandvmap, in float64 or float32, where a tolerance left unset ismax(default, 100 * eps)for the iterate's dtype, and end with aTerminationStatus, among them the newLINE_SEARCH_FAILEDfor a line search that found no acceptable step. A variable whose lower and upper bounds are equal is fixed at its value, as Ipopt'sfixed_variable_treatment="make_parameter"does. See Constrained solvers. - Added
ds.optimization.lanes, which solves a batch of problems that differ inargsandx0, split over the devices of aLayout. Itsmax_steps=,tol=andwarm_start=, like the compiled solvers'run(max_steps=, tol=), are runtime values, one per lane or one for all, so a new value does not compile the loop again. - The compiled solvers differentiate their solution by the implicit function theorem, in
argsand in the problem's bounds: an active bound moves the coordinate it holds, and its multiplier is the objective's rate in it. A tangent on the start point, the warm start or the problem's scales raisesConfigurationError. Withhessian="gauss-newton"on a least-squares problem the rule is the Gauss-Newton form, first order throughout (J^T Jfor the Lagrangian Hessian, no constraint curvature), which equals the exact rule at a zero-residual solution;hessian="exact"or"auto"takes the exact rule. - Added
ds.optimization.control. AControlProblemposes optimal control of a model's operational inputs over a horizon, with a cost, path rows and terminal rows on the solution, and lowers to anOptimizationProblemthatoptsolveand the compiled solvers take.receding_horizonruns the closed loop as one compiled program, andHostLoopsteps it from Python for a plant outside JAX; each horizon starts from an estimator's estimate (FullState,ExtendedKalmanFilter,MovingHorizonEstimator). ds.linear.{KLU,SuperLU,CuDSS}now factor and solve complex matrices (complex128, andcomplex64onCuDSS) on the real symbolic analysis, solve a complex right-hand side on a real factor, and differentiate both ways.- Added
ds.solve_scheduleandds.schedule_arrays, which run aScheduleagainst a model's operating table, withmax_stepsper segment and first and second derivatives with respect to segment durations. See Events and protocols. - Added
ds.optimization.control.SignalControls, which drives aControlProblemwith the time signals ofds.optimization.inference.approx, such as a current on the new clamped cubicBSplineor a current then a voltage, and runs each signal on the schedule exactly. Withfree_durations=Trueeach signal's duration is also a decision variable, andControlProblem(save_fractions=)samples within every segment. - Added a
"resistance"row tospec.control.operationalandSegment(resistance=), which holdvoltage = target * current. - Added
spec.build(exact_sparsity=True), which declares only the Jacobian entries a tangent can reach. - Added
ds.solver_view(problem), the cached view the solvers take of a problem (ds.SolverView), andds.jacobian_nnz(problem), the entry count of its iteration matrix. See Problem interface. - Under
jax.default_device,linear="auto"and consistent initialization choose their backend by that device's platform, and native factors are kept per device, so one process can run lanes on a GPU and on CPUs.checkpointed_map(device_groups=)accepts groups of another platform than the default backend's. KLU.native_statsreportslive_lu_entries.SuperLU.native_statsreportssingular, the factor requests that ended singular, asKLU.native_statsdoes.- Added
ds.jacobian_pattern(fn, y, *args, within=None), the entries of a Jacobian that can be nonzero, read from the function's tangent map without evaluating it. Rules cover loops, branches, gathers, scatters at known indices, cumulative sums, convolutions, linear solves,jax.checkpoint, andjax.custom_vjpfunctions, read through theirbwd; an operation without one falls back to every entry with a warning, bounded bywithin. See Problem interface. ds.topology_reportaddspullback_count, and itscolor_countis now the solver's own: the colors of the rows the solver colors, with the rows it holds out recovered by pullback. Previously it colored the whole pattern.TerminationStatusmembers are now also their integer codes, so a compiled solver'sSolveRecord.statuscompares with them underjitandTerminationStatus(code)reads a code back. Their string values are unchanged.- Added
hessian_composition=toOptimizationProblem.from_valueandLeastSquaresProblem.from_residual. Under"forward-over-forward"the gradient, the Hessians and the Lagrangian Hessianswith_constraintsadds are all forward mode, soInteriorPoint,SQP,AugmentedLagrangianand their implicit derivative take an objective whose inner solve supports forward mode only, such as a DAE solve underForwardAdjoint. The default,"forward-over-reverse", is unchanged. - Added
CheckpointAdjoint(reverse_over_reverse=True), under which a gradient through a solve can itself be differentiated in reverse mode, as injax.gradofjax.grad, and the solve also takesjax.jvp. Forward mode over the gradient then costs about twice as much, so the default,False, keeps the previous rule. - Added
adjoint=toInteriorPoint,SQPandAugmentedLagrangian.ds.nonlinear.ImplicitAdjoint()differentiates the solution inargsby one reverse rule, a transposed KKT solve, sojax.gradcosts one solve however largeargsis and runs through closures that differentiate in reverse mode only. The default,ds.ForwardAdjoint(), keeps the forward rule. - Added
ds.optimization.inference.InequalityRows, rowsc(own, shared, data) <= 0on the fitted values of a lowered study, per member of a level or on the shared values, whichTrustRegion(inequalities=...)keeps. Each step finds its active set by a primal active-set method on the projection that keeps the equality rows, holding a row the step would cross a...
diff-soup 1.1.0
New tutorials and worked examples, and the fixes and features found and added while writing them.
Breaking changes
-
spec.observe.{spatial_mean,boundary_mean}now average.fnreturns one value per node, weighted by cell volume or facet measure. Previously the name was only a label. A scalar-valuedfnis now refused.# Before (1.0.0): fn computed the mean spec.observe.spatial_mean(name="T_avg", fn=lambda q: jnp.mean(q.T)) # After: fn returns nodal values spec.observe.spatial_mean(name="T_avg", fn=lambda q: q.T)
-
spec.bc.neumannnow imposesdu/dn = son FVM, FDM and FEM. Previously it was an alias ofspec.bc.flux, which imposesD du/dn = s.name=andcross_facet=are now onfluxonly.# Before (1.0.0): neumann was an alias of flux spec.bc.neumann("u", on="left", value=s) # imposed D du/dn = s # After: say which one you mean spec.bc.flux("u", on="left", value=s) # D du/dn = s spec.bc.neumann("u", on="left", value=s) # du/dn = s
-
Grid boundary conditions and boundary integrals in 2D and 3D now weight each selected cell by one face. Previously edge and corner cells were counted more than once. A selection spanning several faces now raises. Pick one with
axis=andside=. -
Fixed the sign of flux and Robin terms on clockwise meshes, which includes every
mask_to_meshmesh. Element volumes now use|det J|. A tangled element now raises. -
FEM operators now read a 1D coefficient by its length:
num_dofsvalues are nodal andn_cellsvalues are per cell. Previously a per-cell array was read as nodal. -
The residual form
q.dt(...)now works on meshes (previously its coefficients were dropped), and keeps the boundary conditions of the field it writes on grids and meshes alike. -
NewtonTrustRegionandLevenbergMarquardtnow endSTALLED(converged=False) when the trust radius or damping holds a step short of a stopping test. Previously these fits reported converged.result.info["stall_reason"]says why. -
NewtonTrustRegion(hessian=)now defaults to"auto", which picks Gauss-Newton on any problem with residual rows, including everyInferenceProblem.optsolveandconstrained_optsolvefollow. Previously the default was the true Hessian. Passhessian="true"to get it back. -
NewtonTrustRegion(hessian_tile=)now defaults to"auto", and a true Hessian or Gauss-Newton Jacobian too large for the device stops the fit before its first step with aConfigurationError. Previously the default was"full". -
sample(kernel="nuts")now adapts all chains together (adapt="pooled",warmup=200), and each chain keeps its adapted state. Previously one chain adapted for 500 steps and the others started from a perturbed fit.chains=1, adapt="window", warmup=500reproduces the old draws. -
Identifiability checks now skip network columns, including on multi-experiment problems, so a network-only fit no longer warns.
theta_singular_values,jacobian_condandparam_group_condnow cover the non-network columns only. -
stats.linear_solve_modenow reports"fp32_factor"forPrecision.fp32_factor(). Previously it reported"fp32", which now means end-to-end float32 only.
Features
-
Added
ds.compile, which compiles apdesolvecall without running it. The result runs on newt_span,y0andthetaof the same shapes and dtypes, and reportscompile_seconds,cost_analysis()andmemory_analysis(). See Running a simulation.compiled = ds.compile(problem, solver, (0.0, 0.1), y0) sol = compiled((0.0, 0.2), y0)
-
theta[handle]reads a parameter, andtheta.replace(handle, value)writes every copy of it. A theta that went throughjax.tree.map, a device transfer or a tracer still works withds.default_theta. See Learnable parameters. -
Added
ds.parameter_gradient(grads, handle), which readsdloss/dparamfromjax.grad(loss)(theta)and sums the copies of a parameter that several equations read. -
Added
est.values(x)[handle], which reads parameters from a vector in the fit coordinate.ParameterSchemais exported fromds.inference. -
Added
recipe.spec(**inputs), which returns theProblemSpeca recipe authors, andgrid.{coordinates,meshgrid()}on both grid classes. -
Grid transport:
q.ops.advect_upwindtakes velocity fields,q.ops.div_advectis the conservative form, with flux-limited face values by default (scheme="tvd",limiter="van_leer","koren"or"minmod": second order where smooth, no new extrema at any cell Péclet number), third-order"weno3","central"or first-order"upwind", andq.ops.div_fluxtakes a per-axis(Dx, Dy, Dz).spec.bc.{dirichlet,flux,robin,neumann}takeaxis=andside=, and the newspec.bc.inflowgives the Danckwerts condition withdiv_advect. -
spec.bc.periodic(var, axes=(0,))makes a field periodic on some axes and allows other conditions on the rest. -
Mesh regions:
TaggedMesh(cell_tags=, cell_tag_names=),q.ops.cell_indicator(name),polygons_to_tagged_mesh({name: polygon}, ...)with conforming interfaces and per-region edge tags (segment_tags=,segment_names=), andread_meshphysical groups as cell and boundary tags. -
Mesh boundary conditions: flux, Robin and Neumann conditions,
surface_measureandq.ops.boundary_integralon QUAD4, TRI6, QUAD8, HEX8, TET4, TET10, HEX20 and HEX27. A condition bynode_indicesselects the facets whose vertices all lie in the set. -
On a mesh,
Fieldcomposes with weak terms and nodal arrays and supports**,absand comparisons.c * q.dt(field)takes one value ofcper cell. -
Added
ds.inference.approx.WIRE, a coordinate network with the Gabor-wavelet activation of Saragadam et al. (2023) and the interface ofSIREN. See Function approximation. -
The Gauss-Newton Jacobian is now tiled:
NewtonTrustRegion(hessian_tile=)applies to it, andLevenbergMarquardt(jacobian_tile=)is new. Previously an integerhessian_tilewas ignored under Gauss-Newton. See Solvers. -
NUTS options:
sampletakesadapt=,mass=("diag"or"dense"),step_size=andmax_tree_depth=, anddraws.info["adaptation"]records what ran. See MCMC. -
Added
ds.inference.evidence(est, fit)andlaplace(est, fit).log_evidence, the Laplace evidence of a MAP fit. Its stationarity check scales with the solver tolerance and precision of the problem, it accepts float32 posteriors, and a refusal names the solver tolerance.information,laplace,identifiabilityandpredictnow take a multiple-shooting fit. -
Measurement(observe="name")reads aspec.observedeclaration. -
A fit now seeds
with_start,posterior_target,information(at=),identifiability,profile,predictandOptResult.load(problem=)whenever its parameter layout matches. A rebuilt recipe or another solver tolerance no longer refuses it, and a start can be a mapping{handle: value}. -
verify_gradientnow returns averdict:"passed","failed"or"inconclusive". -
ds.default_thetatakes a whole module for a function parameter.
Bugfixes
ds.default_thetaoverrides no longer leak between two problems equal in content. Fixed parameters declared from one array, and functions declared from one module, are now copied.optsolve(checkpoint=)now resumesLevenbergMarquardtandNewtonTrustRegionfits from their saved steps. It refuses a directory written by a fit of other parameters.sample(kernel="nuts")now audits the gradient at a jittered chain start instead of the MAP, where the audit meant nothing. It refuses only a failed audit.posterior_target(gradient="reverse"),log_posteriorandLogDensity.from_callablenow scale their reverse/forward check with the solverrtoland run it one percent away from the start, so a float32 fit passes at its mode.jax.jitandeqx.filter_jitnow compile aLogDensity.- Fixed
ExperimentSet.log_posterior, which raised on every call. - Meshes: a scalar source term in the residual form now builds, a first use under
jax.jitno longer leaves a tracer in the cache, andread_meshno longer reports TET10 meshes as tangled. - Fixed the operator-level
FluxBC: the wrong sign on the low side of an FVM grid, and a crash with a per-cellDon a 2D grid. It now also works on 2D and 3D FDM grids. - Importing JAX before diff-soup no longer turns off constant hoisting, which made closures over large arrays compile to large programs.
Performance
- Fitted window starts now solve each window once per evaluation instead of once per observed frame. With 110 frames in 5 windows the integrated time drops from 1265 to 110. Fitted values are unchanged up to the solver tolerance.
- With fitted window starts, the Gauss-Newton Jacobian now pushes each direction through all windows at once. On an 803-variable FitzHugh-Nagumo fit one Jacobian takes 7.2 s instead of 31.0 s on CPU, 0.78 s instead of 3.30 s on an RTX 4070 Ti, and 0.33 GiB of compiled memory instead of 1.26 GiB.
Precision.fp32_factor()now solves its float32 factors unrefined, as documented. On Gray-Scott 256x256 atrtol=1e-6on an RTX 4070 Ti, a solve drops from 5.10 s to 3.06 s with the same steps and errors.- Fit diagnostics now differentiate only the non-network columns, which removes most of their cost on network fits.
Other
- Better errors:
jax.gradthrough a solve with the default forward-only adjoint raises aValueErrornamin...
diff-soup 1.0.0
First public release of diff-soup, a differentiable, GPU-native JAX library for stiff PDEs and DAEs.
Wheels for CPython 3.12 and 3.13 on x86_64 Linux (manylinux_2_28). They carry the KLU sparse direct solver; the cuDSS solver is installed with pip install "diff-soup[cu12]" or "diff-soup[cu13]". SHA256SUMS lists the checksums.
Documentation: https://stmorgenstern.github.io/diff-soup/