fix(steady-state): size the finite-difference probes relative to what they perturb (#76) - #122
Merged
Merged
Conversation
… they perturb (#76) `dY_ss/dp = -J^-1*(df/dp)` differences either factor when no closed form is available, and both probes sized their step as `eps*max(|x|, 1)` with `eps = sqrt(machine eps)`. The floor is there to survive `x == 0`, but it also overrides the relative step for everything below 1. On `ode/before_bunching`, whose `KD` is 1e-9 and whose forward rate constant is derived as `kon/KD`, the probe was 1500% of the parameter — `kf` falls from 1.0 to 0.063 across it — so the difference quotient was a secant across the rate law's curvature and `max|dY_ss/dKD|` came back 6.754783e10 against a re-solve oracle's 1.074089e12, stable to 6 figures across three step sizes. 15.9x low. Parameters are now probed at `eps*|p|`; the old absolute step survives only for a parameter that is exactly zero (no scale of its own) or subnormal (no relative step survives the addition, which would divide by zero). Species are probed at `eps*max(|y_j|, max|y|)` — relative to the species, floored at the state's own magnitude rather than at 1.0. Unlike parameters, which have no common unit, every species is a concentration in one, so the state has a typical scale to probe a zero species against; the absolute 1.0 was wrong in both directions, a nanomolar model probed at 1 molar and a molecule-count model probed at 1e-14 of its own state. That scale skips species a `mask=` excluded (#74), whose value is whatever integration left it at and unbounded. Both quotients divide by the realized step `(x + h) - x` rather than the requested one. All four differencing sites take it: the Jacobian and `df/dp` in `compute_ss_sensitivity`, and the state-chain and explicit-parameter halves of `compute_ss_output_sensitivity`. Measured on the 585-model ode_fullnet corpus against the analytical `df/dp` at the same root: of the 850 columns where the step changes and the sensitivity system is not degenerate, those wrong by more than 1e-3 fall 103 -> 57 and 51 cross that line the right way against 5 the wrong way; the 278 control columns with |p| >= 1 move by at most 2.2e-9. Against the analytical Jacobian over 1,066 model-states, the new species rule beats the old one 511 to 54 at 10x and the floor-free relative step 291 to 13. The 5 regressions are the opposite failure mode — a parameter whose own term is a small fraction of the RHS component it sits in loses its response to cancellation at the narrow step — and are filed separately, since closing them needs a step chosen from the measured response.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #76.
dY_ss/dp = -J⁻¹·(∂f/∂p)finite-differences either factor when no closed form isavailable, and both probes sized their step as
eps·max(|x|, 1)witheps = sqrt(machine eps)= 1.49e-8. The floor is there to survivex == 0, but italso overrides the relative step for everything below 1, and then the probe is no
longer small compared with what it perturbs.
The reported case
ode/before_bunchingcarriesKD = 1e-9with the forward rate constant derivedas
kon/KD, so the probe is 1500% of the parameter — it dragskffrom 1.0to 0.063, and the quotient is a secant across a decade and a half of the rate
law's curvature.
max abs dY_ss/dKDh/p= 1e-3 / 1e-4 / 1e-5)The fix
eps·|p|. The old absolute step survives only for aparameter that is exactly zero (no scale of its own) or subnormal (no relative
step survives the addition, which would divide by zero). Parameters have no
common unit, so
|p|is the only scale the model offers.eps·max(|y_j|, max|y|): relative to the species, floored at thestate's magnitude rather than at 1.0. Unlike parameters, every species is a
concentration in one unit, so the state has a typical scale to probe a zero
species against. The absolute 1.0 was wrong in both directions — a nanomolar
model probed at 1 molar, and a molecule-count model (~1e6) probed at 1e-14 of
its own state, which is cancellation noise, not a derivative.
mask=excluded (steady_state(): no way to exclude write-only accumulator species from the convergence test #74): a write-onlyaccumulator holds whatever integration left it at (7.5e8 on Barua 2013 against
1e-10 for the settled species) and would otherwise set the probe for everyone
else.
(x + h) - x, not the requestedone.
All four sites take it: the FD Jacobian and FD
∂f/∂pincompute_ss_sensitivity,and the state-chain and explicit-parameter halves of
compute_ss_output_sensitivity's∂func/∂pfallback.Blast radius (the issue asks for this before picking a fix)
The FD
∂f/∂pis reachable fromSimulator.steady_state(sensitivity_params=…)only when codegen emits no
bngsim_codegen_sens_rhs— since #67/#89 that isMichaelis-Menten and Functional laws carrying a condition or a non-smooth
builtin: 41 of the 585
ode_fullnetmodels. 15 of those also carry aparameter the floor probes at more than 0.1% of its value (up to 1.27e30 for
ode/AVdyn6'sepsilon= 1.18e-38). Drivingfind_steady_statewithout acodegen artifact — how the repo's own tests reach the fallback, and how
before_bunchingwas found — exposes any of the 267 corpus models carryingsuch a parameter. The species probe is reachable wherever the analytical Jacobian
is incomplete, and through
jacobian="fd"everywhere.Those 15 do not, today, yield a case the re-solve oracle can score: 11 never
reach a steady state at all (
tol=1e-11,max_time=1e7), and on the rest thesmall parameters do not move the root (truth ≡ 0) or the re-solve is not stable
in
h. So the quantitative evidence below comes from the corpus A/B and fromclosed-form models, not from those 15.
Corpus A/B: FD
∂f/∂pagainst the analytical one, at the same rootEvery model solved twice — once with the compiled
∂f/∂p, once forced onto thefallback — over its four smallest parameters plus the largest as a control.
1,201 columns over 358 models that converge and have an analytic column to be
scored against. Errors are normalized by the model's sensitivity scale, not
the column's own:
dimer_phos2'sdY/dNApeaks at 1.9e-41 against 1.1e-05 forthe same model's
dY/dV, and normalizing per column makes that dust count asmuch as the real answer.
Of the 850 columns where the step rule changes (
|p| < 1) and the sensitivitysystem is not itself degenerate (
sens_jacobian_rcond > 1e-8):The 278
|p| >= 1control columns move by at most 2.2e-9 (the realized-steprounding), which is what confirms the change is confined to where the step
differs. The largest class of fixes is compartment volumes: 37 columns over 35
models —
Vcyt/V/Vecfat 1e-12, probed at 15,000x themselves — were100% wrong and now land within 1e-5.
The regressions are real and I am not hiding them. Where a parameter's own
term is a tiny fraction of the RHS component it sits in, a probe relative to the
parameter moves
fby less than its roundoff, where the old wide step did not.Worst case
FceRI_viz'skp1 = 1.7e-6, 4.2e-07 → 5.0e-01. Fixing that needs astep chosen from the measured response rather than from
|p|alone — filed as#123 rather than folded in here, because it is a different algorithm
(probe, measure the RHS's own jitter, widen per component) and wants its own
validation.
Corpus A/B: the species probe against the analytical Jacobian
1,066 model-states (every corpus model with a complete analytical Jacobian, at
its initial condition and at a generic all-species-populated state), scored on
‖J_fd − J_an‖∞ / ‖J_an‖∞:Head to head at 10x, the shipped rule is beaten 511 to 54; the floor-free
relative step is beaten 291 to 13 — the floor is what keeps a species far below
the state's scale out of the cancellation noise.
Tests
python/tests/test_codegen_steady_state_fd_step.py, 7 tests, 6 fail onmain(the seventh is the control asserting the well-scaled parameters did not move).
Three new fixtures, each with its closed-form steady state and closed-form
gradient derived in the
.netheader, so the assertions owe the solver nothing:reciprocal_rate_const.net—A <-> Bwithkf = kon/KD,KD = 1e-9(
before_bunching's shape, and the reason the RHS is non-linear inKD— abare mass-action constant is linear, so any step is exact for it).
dA*/dKD = 2.5e8; the shipped step returns 1.572e7, off by 93.7%. ItsperKD() = A_tot/KDfunction carries the same defect intocompute_ss_output_sensitivity:-1.572e16against-2.5e17exact.nanomolar_dimer.net—A + A <-> BatA* = 1e-9underjacobian="fd".Homodimerization is quadratic in A, so the species step does not cancel out of
the Jacobian column; the shipped floor leaves a spurious
-kdim·hworth 7.5xthe real term and the gradient comes back 78.8% low.
nanomolar_dimer_sink.net— the same model plus a write-only accumulator at1e3, masked out. Pins that a species with no steady value does not set the
probe scale for the species that have one.
The file is added to
mir.yml's run list (it matches the existingtest_codegen*.pypath filter, but that list is explicit — see the note in thatworkflow).
Full
python/testssuite green locally against the patched core: 2982 passed, 114 skipped.