Repository navigation
Sparsity read from the equations, compiled constrained optimization in ds.optimization, and control problems in ds.optimization.control, with complex linear solves on every exact backend, a public schedule solve, and fixes to JIDA and consistent initialization.
Breaking changes
TrustRegion(nested_at=...)now takes the joint fit's step and, after an accepted step, polishes each member's own values with the outer values held, on lane passes that reuse the last Jacobian; withinner=TrustRegion(max_steps=0)it is the joint fit. Previously its outer step was the trust-region step of the Schur complement into the shared block, each member's response bounded by its own radius, and the polish ran before the outer acceptance. Its stop rules and verdict are now the joint fit's, and a lane that solves at an accepted trial but fails in theblocksevaluation that ends the members' polish raisesValueErrornaming the lane and ends the fit, as the joint fit does at an accepted point; previously that trial was rejected.- Removed
diff_soup.problem.implicit.get_or_build_solver_view. Useds.solver_view, which returns the same cached view. - Removed
ds.inference.constrained_optsolveandds.inference.InequalityConstraints. Declare the constraints asds.optimization.Constraintson anInferenceProblemand solve it withds.optimization.InteriorPointords.optimization.AugmentedLagrangian. - JIDA with algebraic rows out of the error test, its stock setting, now takes a second Newton iteration when a step's first correction exceeds weighted norm 1. Previously a stale convergence rate could accept it, leaving an algebraic state far off its constraint and its gradients wrong. Results on DAEs change at the level of the solver tolerance.
- Consistent initialization now accepts a final residual within
ic_atol + ic_rtol * S, withS = ‖|F_y| |y| + |F_ydot| |ydot|‖the size of the residual's terms at the final iterate, plus the rounding bound of its longest row, after Newton has tried the absolute target. Previously a start at the rounding floor of large states failed withINCONSISTENT_IC.ToleranceSpec(ic_rtol=0.0)keeps the absolute gate, as doesic_atol=withoutic_rtol=, and so does an engine built withic_tol=and noic_rtol=, which previously took the step'srtol. - While a crossing is located,
Eventconditions now receive the solve'sthetaasargs, as they do when the event is detected, so a condition can read and differentiate a parameter. Previously they receivedNonethere, and a condition that testsargs is Nonenow takes the other branch. spec.build()now declares only the Jacobian entries a tangent can reach, unless an equation usesq.dt(exact_sparsity=None, the new default). Passexact_sparsity=Falsefor the declared blocks with a dense border per global.- A
SolveContextnow belongs to the thread that first factors in it, whatever the backend:factorandempty_factorin it from another thread, or a trace there of a function that calls them, raiseFactorContractError. Create one context per thread. TrustRegionon a lowered study now compiles atrial_predicateand a custom regularization into each attempt. One that readsxon the host now raises. Previously both ran in Python between the attempt's compiled calls.
Features
- Added
ds.optimization.InteriorPoint,SQPandAugmentedLagrangian, compiled solvers for problems with bounds and with equality and inequality rows declared asds.optimization.Constraintson the problem. They run underjitandvmap, in float64 or float32, where a tolerance left unset ismax(default, 100 * eps)for the iterate's dtype, and end with aTerminationStatus, among them the newLINE_SEARCH_FAILEDfor a line search that found no acceptable step. A variable whose lower and upper bounds are equal is fixed at its value, as Ipopt'sfixed_variable_treatment="make_parameter"does. See Constrained solvers. - Added
ds.optimization.lanes, which solves a batch of problems that differ inargsandx0, split over the devices of aLayout. Itsmax_steps=,tol=andwarm_start=, like the compiled solvers'run(max_steps=, tol=), are runtime values, one per lane or one for all, so a new value does not compile the loop again. - The compiled solvers differentiate their solution by the implicit function theorem, in
argsand in the problem's bounds: an active bound moves the coordinate it holds, and its multiplier is the objective's rate in it. A tangent on the start point, the warm start or the problem's scales raisesConfigurationError. Withhessian="gauss-newton"on a least-squares problem the rule is the Gauss-Newton form, first order throughout (J^T Jfor the Lagrangian Hessian, no constraint curvature), which equals the exact rule at a zero-residual solution;hessian="exact"or"auto"takes the exact rule. - Added
ds.optimization.control. AControlProblemposes optimal control of a model's operational inputs over a horizon, with a cost, path rows and terminal rows on the solution, and lowers to anOptimizationProblemthatoptsolveand the compiled solvers take.receding_horizonruns the closed loop as one compiled program, andHostLoopsteps it from Python for a plant outside JAX; each horizon starts from an estimator's estimate (FullState,ExtendedKalmanFilter,MovingHorizonEstimator). ds.linear.{KLU,SuperLU,CuDSS}now factor and solve complex matrices (complex128, andcomplex64onCuDSS) on the real symbolic analysis, solve a complex right-hand side on a real factor, and differentiate both ways.- Added
ds.solve_scheduleandds.schedule_arrays, which run aScheduleagainst a model's operating table, withmax_stepsper segment and first and second derivatives with respect to segment durations. See Events and protocols. - Added
ds.optimization.control.SignalControls, which drives aControlProblemwith the time signals ofds.optimization.inference.approx, such as a current on the new clamped cubicBSplineor a current then a voltage, and runs each signal on the schedule exactly. Withfree_durations=Trueeach signal's duration is also a decision variable, andControlProblem(save_fractions=)samples within every segment. - Added a
"resistance"row tospec.control.operationalandSegment(resistance=), which holdvoltage = target * current. - Added
spec.build(exact_sparsity=True), which declares only the Jacobian entries a tangent can reach. - Added
ds.solver_view(problem), the cached view the solvers take of a problem (ds.SolverView), andds.jacobian_nnz(problem), the entry count of its iteration matrix. See Problem interface. - Under
jax.default_device,linear="auto"and consistent initialization choose their backend by that device's platform, and native factors are kept per device, so one process can run lanes on a GPU and on CPUs.checkpointed_map(device_groups=)accepts groups of another platform than the default backend's. KLU.native_statsreportslive_lu_entries.SuperLU.native_statsreportssingular, the factor requests that ended singular, asKLU.native_statsdoes.- Added
ds.jacobian_pattern(fn, y, *args, within=None), the entries of a Jacobian that can be nonzero, read from the function's tangent map without evaluating it. Rules cover loops, branches, gathers, scatters at known indices, cumulative sums, convolutions, linear solves,jax.checkpoint, andjax.custom_vjpfunctions, read through theirbwd; an operation without one falls back to every entry with a warning, bounded bywithin. See Problem interface. ds.topology_reportaddspullback_count, and itscolor_countis now the solver's own: the colors of the rows the solver colors, with the rows it holds out recovered by pullback. Previously it colored the whole pattern.TerminationStatusmembers are now also their integer codes, so a compiled solver'sSolveRecord.statuscompares with them underjitandTerminationStatus(code)reads a code back. Their string values are unchanged.- Added
hessian_composition=toOptimizationProblem.from_valueandLeastSquaresProblem.from_residual. Under"forward-over-forward"the gradient, the Hessians and the Lagrangian Hessianswith_constraintsadds are all forward mode, soInteriorPoint,SQP,AugmentedLagrangianand their implicit derivative take an objective whose inner solve supports forward mode only, such as a DAE solve underForwardAdjoint. The default,"forward-over-reverse", is unchanged. - Added
CheckpointAdjoint(reverse_over_reverse=True), under which a gradient through a solve can itself be differentiated in reverse mode, as injax.gradofjax.grad, and the solve also takesjax.jvp. Forward mode over the gradient then costs about twice as much, so the default,False, keeps the previous rule. - Added
adjoint=toInteriorPoint,SQPandAugmentedLagrangian.ds.nonlinear.ImplicitAdjoint()differentiates the solution inargsby one reverse rule, a transposed KKT solve, sojax.gradcosts one solve however largeargsis and runs through closures that differentiate in reverse mode only. The default,ds.ForwardAdjoint(), keeps the forward rule. - Added
ds.optimization.inference.InequalityRows, rowsc(own, shared, data) <= 0on the fitted values of a lowered study, per member of a level or on the shared values, whichTrustRegion(inequalities=...)keeps. Each step finds its active set by a primal active-set method on the projection that keeps the equality rows, holding a row the step would cross at its boundary and releasing a held row whose multiplier is negative, and a trial that crosses a row is moved back to it.bootstraprefuses asolverwithinequalities=, and with its default solver a fit made under them, which records its row count ininfo["inequality_rows"]. See Optimization solvers. - Added the
"frozen"mode of a lowered study: afterprepare(modes=(..., "frozen"))eachblocksevaluation keeps the Jacobians of its lanes that solved, and a"frozen"evaluation returns each lane's cost from a plain solve and its gradient on the kept Jacobian, at the cost of avalueevaluation. The nestedTrustRegion's polish runs on it; on a CPU the nested fit cuts itsblocksandfrozencalls at the members' boundaries, so a polish request runs only the members still in it. TrustRegion(constraints=...)andTrustRegion(gauges=...)now also take oneLinearConstraintor oneGauge, asinequalities=takes oneInequalityRows.
Bugfixes
- Fixed MAP fits of a
@ds.recipetemplate, which took every prior from the template's first build in the process, so a later build with another prior was fitted under the first one's. Each fit now takes each prior from its own experiments' builds (per owner for a parameter that varies by owner), aGaussianCoefficientPriorcenters on its own build's coefficients, builds that differ only in a prior return two problems, and a MAP fit refuses a parameter shared by experiments whose builds declare different priors. - Fixed member equations, whose
q.p.<name>andq.fn.<name>raisedAttributeError. They now read the problem's declared parameters and functions, with their tangents, as component equations do. - Fixed consistent initialization of a family problem whose step backend is a
MemberBlockover the full state, which raised aConfigurationError. Initialization now solves its smaller system on the backendlinear="auto"chooses for its size. - Fixed
SaveAt(output="projected")rows that came backNaNat a tightatol. Projected samples now pass the same gate as the solve's own initialization. - Fixed JIDA solves that stalled at a tight
atolafter a short step, where Newton's corrections are pure rounding. A correction within twice the rounding floor on a fresh matrix is now accepted as the root. - Fixed the derivative of consistent initialization with respect to the start time.
- Fixed
TrustRegionwithconstraints=orgauges=from a start off the equality rows, which could return the start as converged when the step onto the rows raised the cost. The start now moves onto the rows first, by the least change of its free coordinates. - Fixed
TrustRegionon a lowered study, whose radius collapsed after a trial of zero length. Such a trial now keeps the radius. - Fixed
jax.gradintofCubicHermiteandPiecewiseLinearon a knot, which was half the derivative of the segment the knot selects. It is now the whole one: the knot's slope forCubicHermite, and forPiecewiseLinearthe slope of the segment to its right (of the last segment on the last knot). - Fixed
SolveContext.resident_bytesonKLU, which read uninitialized memory for any factor with a singleton block. - Fixed a MAP fit of a joint problem, which read every shared parameter's prior from one handle. Each occurrence now takes its prior from the declarations of the experiments behind it, and a MAP refuses an occurrence whose schema records none.
- A
KLU,SuperLUorCuDSSfactor lowered into a program over several devices of one process now raisesFactorContractError. Previously XLA failed with an internal error, or, in ashard_mapbody, the devices shared one native factor. - Complex values on an analysis declared
spd=True, and on aCuDSSanalysis withdowncast_factorsoremulation_mantissa_bits, now raiseFactorContractError. Previously cuDSS factored them as a real symmetric positive definite matrix, or ignored the option. - Fixed
SuperLUon more than one thread, which returned a wrong solution for a few factors in a hundred with a success status. Its factors on any number of threads now equal one thread's, bit for bit; the build patches SuperLU_MT 4.0.2 (diff_soup/_internal/superlu_ffi/patches/). - Fixed
CuDSSsolves that could apply another factor and report success: two executions of one compiled program at once on oneSolveContext, or two calls of one jitted function inside one program, which shared a factor slot. Each call now keeps its own factor, and a solve always applies its own handle's factor. - A released
AnalysisofKLU,SuperLUorCuDSSnow frees its native analysis, its call sites and its factors in everySolveContext,DEFAULT_CONTEXTincluded, once no compiled program that factors with it exists. Previously the sites kept every analysis for the life of the process. - Fixed a
SuperLUsolve with real values of a shared storage that a refactor with complex values filled earlier in the same program, which returnednanwith a success status. It now factors the real values first, asKLUdoes.
Performance
- A vmapped solve keeps fewer copies of its saved output (two state rows per save time, from five), and
ds.topology_reportno longer builds an n-by-n adjacency for a problem with a full row. An LFP half cell with 206,518 states and 1,001 saved times now completes on a 12 GB GPU. - JIDA does its whole-array work only on the path an attempt takes, with identical results.
- Outside
vmap, the branches of an implicit step (JIDA's history update, the Jacobian refreshes, the retries) compile to one conditional and are traced once. Undervmapa branch still runs only when some lane takes it, with one compiled copy of each branch. - Consistent initialization's derivative skips the start-time term when the start time is constant, and its relative gate evaluates the residual's scale only where the absolute target was missed, on the algebraic rows alone on the reduced route.
spec.build(exact_sparsity=True)reads its pattern with one compiled program instead of one per operation, 0.85 s instead of 3.6 s at 18 families, and neither build colors the declared pattern unlessphysics_jacobian_blocksis called. Loops, cumulative sums, convolutions and linear solves in a residual keep their pattern instead of filling it.- Member rows that read every site of their member are recovered by shared pullbacks instead of one color per site: an LFP half cell with 50 volumes of 10 particles needs 8 colors instead of 108.
TrustRegionon a lowered study now runs each attempt as two compiled programs, one that proposes the trial and one that judges it, with the MAP priors' rows, the trial predicate and the active set ofinequalities=inside them; withnested_at=, each polish iteration is two more, for all members at once. Fits of one lowered study share their compiled programs, so a second fit compiles nothing: a second joint fit of a 192-experiment study takes 0.10 s on a CPU, from 2.4 s. Atrial_predicateor a custom regularization that readsxon the host now raises when it is traced.- Inside one compiled program, each call of a jitted function that factors on
KLUorSuperLUnow keeps its own factor: k calls refactor k times a run, from 2k.
Other
- Three undocumented module paths moved:
diff_soup.timestepping.consistent_icanddiff_soup.problem.consistent_initare nowdiff_soup.problem.implicit.consistent_init, anddiff_soup.timestepping.vsvo.common.weightsis nowdiff_soup.utils.weights. ds.optimization.inference.approx.FluxNetraisesAttributeErrorlike any other missing name. It raised aTypeErrornamingeqx.nn.MLPsince its removal.CuDSS'scontract_statsno longer reportsperturbed_pivot_factorizations, which was always 0: every cuDSS factor runs with pivot perturbation off.- The cuDSS extension's default CUDA architectures add
87-real(Jetson Orin). - The wheels now include SuperLU_MT, so
linear="superlu"needs no source build. - Added wheels for Linux on ARM (aarch64).
- A diff-soup environment variable set to an empty string now counts as unset and takes its default. Previously an empty
DIFFSOUP_PROCESSESorDIFFSOUP_NODESraisedValueError. diff_soup.inferenceis nowdiff_soup.optimization.inference, andoptsolve,OptResult, the optimization problems and the solvers are also at the top ofds.optimization.ds.inferenceand everydiff_soup.inferenceimport path still work for this release, return the same objects and warn once with aDeprecationWarning; they will be removed in 1.3.
Full Changelog: v1.1.0...v1.2.0
Wheels for CPython 3.12 and 3.13 on Linux x86_64 and aarch64 (manylinux_2_28). They carry the KLU and SuperLU_MT sparse direct solvers. Install the cuDSS solver with pip install "diff-soup[cu12]" or "diff-soup[cu13]". SHA256SUMS lists the checksums.
Documentation: https://stmorgenstern.github.io/diff-soup/
Full Changelog: v1.1.0...v1.2.0