Skip to content

Gradient fit aborts with AttributeError: NoneType.items when a simulation fails mid-fit (gradient_at doesn't guard res.simdata is None) #492

Description

@wshlavacek

Summary

A gradient fit (job_type = lbfgs / trf) crashes with AttributeError: 'NoneType' object has no attribute 'items' when the ODE simulation fails at a candidate point during the fit. gradient_at assumes every result carries simdata, but a failed simulation leaves res.simdata is None, and the gradient assembly dereferences it unconditionally.

Environment

  • PyBNF main (c62d9213)
  • bngsim 0.11.35, SBML model via sbml_backend = bngsim

Traceback

The proximate cause is a simulation failure that is then dereferenced:

bngsim._exceptions.SimulationError: Simulation failed:
    CVODE integration failed at t=4.000000 with flag=-4
...
  File "pybnf/algorithms/base.py", line 1086, in _record_result_and_decide
    response = self.got_result(res)
  File "pybnf/algorithms/optimizers/gradient_base.py", line 270, in got_result
    grad = self.gradient_at(res)
  File "pybnf/algorithms/optimizers/gradient_base.py", line 458, in gradient_at
    for model_name, by_suffix in res.simdata.items():
                                 ^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'items'

Root cause

When bngsim's CVODE integration fails at a candidate parameter point (a stiff/extreme point in the search), the job returns with res.simdata = None rather than trajectory data. gradient_at (gradient_base.py:458) iterates res.simdata.items() with no None guard, so the whole fit aborts instead of treating that point as a failed evaluation.

The metaheuristic path handles a failed simulation as a non-finite/rejected objective; the gradient path does not. This is the same res.simdata is None failure mode that #480 guarded in the sampler / constraint-tracking path — the gradient path has the analogous unguarded case.

Why it looks intermittent (start-point dependent)

It surfaces only at candidate points where CVODE fails, so it depends on where the optimizer starts/steps. On the PEtab benchmark collection it flips between problems depending on the box-center start: e.g. Crauste_CellSystems2017 fails from its log-scale start while Elowitz_Nature2000 fails from its linear start (and each succeeds from the other). One bug, surfaced by whichever start lands on a non-integrable point.

Reproduction

Import a PEtab SBML problem whose gradient search passes through a non-integrable point and run a gradient fit with sbml_backend = bngsim, e.g. Crauste_CellSystems2017 (lbfgs, population_size = 1, max_iterations = 2).

Suggested fix

In the gradient path (got_result / gradient_at), treat res.simdata is None (a failed simulation) as a failed evaluation — skip the gradient and return a non-finite objective / rejected-step sentinel — mirroring the scalar path's handling of a failed simulation and the res.simdata is None guard added in #480. A gradient optimizer should back off (shrink the trust region / reject the step), not abort the fit, when a candidate point does not integrate.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions