feat(codegen): analytic df/dp for Functional rate laws, emitted and gated off (#66) - #83
Merged
Merged
Conversation
…ated off (#66) generate_sens_from_model returned None on the first non-Elementary reaction, so one Functional rate law out of 689 put a whole model on CVODES' internal difference quotient. This derives the missing half. The `!= "elementary" -> return None` gate stays shut: the new path is reachable only through a keyword-only `functional=` that defaults to False and that no production caller sets, so nothing changes at runtime. #67 flips it. The maths needed no emitter surgery. A Functional rate is stat_r * func_r(obs, p, t) * [af_r * prod R_r], the species factor is parameter-free, and an observable is a fixed linear combination of species (d obs/dp = 0 at fixed y), so df_i/dp = sum_r stat_r * netstoich_ir * (dfunc_r/dp) * af_r * prod R_r is the same "(derivative expression) x geometry" product _emit_sens_rhs_body already emits for a derived rate constant (#15/#41). The terms ride that existing derived_terms channel with dfunc_r/dp in place of dp_d/dprimary; what is new is only the differentiation, and only ~90 lines of it, because _inline_functions, _exprtk_to_sympy, sympy_to_c and _derived_param_jacobian_checked all apply as-is. A derived parameter is deliberately NOT inlined into the rate law: it keeps its own column as the direct partial (matching how the Elementary path emits case iP for a _rateLaw{N}), and the chain rule to its primaries is added as extra terms through _derived_param_jacobian_checked, which is also what keeps a *nested* derived parameter reachable. Decline loudly, per the #56 precedent and because CVodeSensInit1 takes one callback for every column, so a decline is necessarily model-wide. Every refusal names what blocked it, and an empty term list stays a *success* (a genuinely zero column) rather than being confused with a failure — that ambiguity is what #56 was filed about. The construct gate reuses _jacobian.unsupported_expr_construct rather than re-spelling it; two spellings drift, which is exactly how the #53/#56 hole survived a release. The conditional class it rejects is #68's to lift, and only behind the shared switch-time guard. One decline the per-reaction loop cannot see coming — every law differentiates but the obs[]/func[] values they are written in need data->tfun_eval, which CodegenSensUserData does not have (#65) — gets its own warning rather than a quiet None. Oracle, solver-free and corpus-wide: a central finite difference of the *emitted* bngsim_codegen_rhs against the emitted df/dp, both through ctypes, with a primary-parameter perturbation going through set_param so the derived-parameter chain rule is under test rather than assumed. Three well-separated steps, since cancellation noise falls as 1/h while truncation grows as h^2; a first pass at one step reported its own noise as codegen bugs (worst offender: an RHS with ||f||_inf = 3.4e7, where the FD error falls monotonically from 1.0 at h=2e-8 to 9.5e-7 at h=2e-2 against an analytic value of exactly 1). Measured on the 585 .net files under benchmarks/suites/ode_fullnet/nets: 400 all-Elementary (untouched) | 185 non-Elementary 137 emitted, 5315 dfunc/dp terms | 48 declined | 0 errors 14127 (parameter, state) columns differenced; worst disagreement 0.22 of the per-component tolerance, i.e. every emitted model agrees to FD precision declines: 38 if() | 6 max() | 2 min() | 1 derived param whose own flattened expression carries a min() | 1 Michaelis-Menten (out of scope, worth exactly one extra model) Derivation cost is 2.6 s for all 185 models, 0.31 s worst case, memoized on the rate-law text since a rule-generated network reuses one expression across many reactions. The no-behavior-change claim is verified rather than argued: generate_sens_c, generate_sens_from_model and the full combined source are byte-identical to main on all 585 models, and the 189-model fallback set is unchanged. Not covered, and declined rather than approximated: cross-compartment (per-species volume scaling) Functional reactions, whose df/dp scatter would need the per-row (possibly live, #171) volume divide the emitted switch has no form for. No .net in the corpus is one. Note for #67: bngsim_jac_vec is still Elementary-only, so a Functional model's bngsim_codegen_sens_rhs is INCOMPLETE — the emitted C says so in its header. Stage 2 of #55. Depends on #65. Blocks #67.
#66) Three of the five decline paths built a reason string and dropped it: a Michaelis-Menten (or otherwise unknown) rate-law type, a Functional reaction whose function name resolves to no expression, and a cross-compartment per-species-volume-scaling reaction all returned ({}, reason) while only the differentiation failures warned. generate_sens_from_model discards the reason, so those three were silent declines — the exact shape #56 exists to prevent, just one level up from a silent zero. _functional_dfdp_terms now returns through one _decline() helper that warns and returns in the same step, so a new decline cannot be added without a message. The warning helper drops its (rxn_label, expr) parameters for a single reason string, since the three model-level declines have no rate-law expression to quote and the per-reaction ones fold the label and the expression into the reason anyway. Corpus unchanged: 137 emitted / 48 declined / 0 errors, worst FD ratio 0.2248, 5315 terms; and still byte-identical to main on all 585 models with the 189-model fallback set unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #66. Stage 2 of #55, on top of #65 (
obs[]/func[]plumbing). Blocks #67.What this is
generate_sens_from_modelreturnsNoneon the first non-Elementary reaction,so one Functional rate law out of 689 puts a whole model on CVODES' internal
difference quotient. This derives the missing half.
Nothing changes at runtime. The
!= "elementary" → return Nonegate staysshut: the new path is reachable only through a keyword-only
functional=thatdefaults to
Falseand that no production caller sets. #67 flips it.The derivative needed no emitter surgery
A Functional rate is
stat_r · func_r(obs, p, t) · [af_r · ∏R_r], the speciesfactor is parameter-free, and an observable is a fixed linear combination of
species (
∂obs/∂p = 0at fixedy), sois the same (derivative expression) × geometry product
_emit_sens_rhs_bodyalready emits for a derived rate constant (#15/#41). The terms ride that existing
derived_termschannel with∂func_r/∂pin place of∂p_d/∂primary. What isnew is only the differentiation — ~90 lines, because
_inline_functions,_exprtk_to_sympy,sympy_to_cand_derived_param_jacobian_checkedall applyas-is, exactly as #55's reuse table predicted.
A derived parameter is deliberately not inlined into the rate law: it keeps
its own column as the direct partial (matching how the Elementary path emits
case iPfor a_rateLaw{N}), and the chain rule to its primaries is added asextra terms — which is also what keeps a nested derived parameter reachable.
The SIR exemplar,
betaI() = beta*Iwithbeta = 1/S0 # ConstantExpression:Decline loudly
CVodeSensInit1takes one callback for every column, so a decline is necessarilymodel-wide — there is no per-reaction fallback to mix in. Every refusal names
what blocked it, and an empty term list stays a success (a genuinely zero
column) rather than being confused with a failure; that ambiguity is what #56 was
filed about.
The construct gate reuses
_jacobian.unsupported_expr_constructrather thanre-spelling it. Two spellings drift, which is exactly how the #53/#56 hole
survived a release. The conditional class it rejects is #68's to lift, and only
behind the shared switch-time guard.
One decline the per-reaction loop cannot see coming — every law differentiates,
but the
obs[]/func[]values they are written in needdata->tfun_eval, whichCodegenSensUserDatadoes not have (#65) — gets its own warning rather than aquiet
None.Oracle: solver-free, corpus-wide
Central finite difference of the emitted
bngsim_codegen_rhsagainst theemitted
∂f/∂p, both through ctypes,yS = 0soJ·ySvanishes exactly. Aprimary-parameter perturbation goes through
set_param, which re-derives everyConstantExpression parameter — so the #15/#41 chain rule is under test rather
than assumed.
Three well-separated steps, because cancellation noise falls as
1/hwhiletruncation grows as
h². This is not belt-and-braces: a first pass at one stepreported its own noise as codegen bugs. Worst offender, an RHS with
‖f‖∞ = 3.4e7where the analytic answer is exactly1:Monotone
1/hdecay — the textbook noise signature, not a wrong derivative.Measured: 585
.netfiles underbenchmarks/suites/ode_fullnet/nets∂func/∂pterms14 127 (parameter, state) columns differenced. Worst disagreement 0.22 of the
per-component tolerance — every emitted model agrees to FD precision, with a
4.4× margin at the worst point.
Declines: 38
if()· 6max()· 2min()· 1 derived parameter whose ownflattened expression carries a
min()· 1 Michaelis–Menten (out of scope; worthexactly one extra model). The 137/48 split matches the 140 CLEAN / 45 blocked
partition #55's staging measured independently.
Derivation cost: 2.6 s for all 185 models, 0.31 s worst case, memoized on the
rate-law text since a rule-generated network reuses one expression across many
reactions.
The no-behavior-change claim is verified, not argued
generate_sens_rhs_c,generate_sens_from_modeland the full combined sourceare byte-identical to main on all 585 models, and the 189-model fallback set
is unchanged (396 models get a sens RHS before and after).
test_codegen_functional_dfdp.pypins the same claim per-model, including that an Elementary model's #15 derived
chain is untouched with the keyword set.
Not covered, and declined rather than approximated
Cross-compartment (per-species volume scaling) Functional reactions: their
∂f/∂pscatter would need the per-row (possibly live, #171) volume divide theemitted switch has no form for. No
.netin the corpus is one.The
.netemittergenerate_sens_rhs_cis deliberately untouched, for #65'sreason: the
.netparse carries neither the observable-entry nor thetable-function shapes
_emit_observable_lines/_emit_function_linesconsume,which is also why
generate_combined_calready sources its Jacobian from themodel. Michaelis–Menten stays out of scope — 11 reactions across 2 models, one
of which is Functional-blocked anyway, so it unblocks exactly one extra model.
For #67
bngsim_jac_vecis still Elementary-only, so a Functional model'sbngsim_codegen_sens_rhsis incomplete — the emitted C says so in its ownheader, not only in the Python docstring:
Testing
python/tests/test_codegen_functional_dfdp.py(gate-shut,byte-identity, emitted terms, seven decline classes, four FD-oracle models).
ruff,ruff-format,mypyclean.