Optimistic first time convergence criterion

[YingboMa]

CVODE estimates the convergence of the first iteration by reusing the
convergence rate of the previous nonlinear solver iteration.

Reference: https://github.com/LLNL/sundials/blob/2abd63bd6cbc354fb4861bba8e98d0b95d65e24a/src/cvodes/cvodes_nls.c#L325-L331

MWE: https://discourse.julialang.org/t/cvode-bdf-outperforms-julia-solvers-for-stiff-system-biology-model/113936

Master branch:

julia> sol = solve(oprob_jac, FBDF()).stats
SciMLBase.DEStats
Number of function 1 evaluations:                  311
Number of function 2 evaluations:                  0
Number of W matrix evaluations:                    31
Number of linear solves:                           222
Number of Jacobians created:                       2
Number of nonlinear solver iterations:             212
Number of nonlinear solver convergence failures:   0
Number of fixed-point solver iterations:                     0
Number of fixed-point solver convergence failures:           0
Number of rootfind condition calls:                0
Number of accepted steps:                          84
Number of rejected steps:                          2

This PR:

julia> sol = solve(oprob_jac, FBDF()).stats
SciMLBase.DEStats
Number of function 1 evaluations:                  251
Number of function 2 evaluations:                  0
Number of W matrix evaluations:                    32
Number of linear solves:                           159
Number of Jacobians created:                       2
Number of nonlinear solver iterations:             149
Number of nonlinear solver convergence failures:   0
Number of fixed-point solver iterations:                     0
Number of fixed-point solver convergence failures:           0
Number of rootfind condition calls:                0
Number of accepted steps:                          86
Number of rejected steps:                          3

[ChrisRackauckas]

CVODE estimates the convergence of the first iteration by reusing the convergence rate of the previous nonlinear solver iteration.

wild! I wonder how that effects SDIRK.

[YingboMa]

Yeah, this should make SDIRK methods more competitive against BDF methods and Rosenbrock methods as well.

[YingboMa]

Here's the evolution of the convergence rate for this particular problem. I guess constant extrapolation isn't a bad guess to capture the plateaus.

[YingboMa]

Here's theta vs model time.

[YingboMa]

The test failures seem unrelated.

[devmotion]

AFAICT the DelayDiffEq test failures are caused by this PR.

[ChrisRackauckas]

type FPSolver has no field prev_θ

Yes I'm very sure that's related to this PR.

[ChrisRackauckas]

For reference, only Interface I is failing on master, and I hope to get to that this weekend.

[YingboMa]

Many of the regression tests might be too stringent.

[ChrisRackauckas]

https://github.com/SciML/OrdinaryDiffEq.jl/actions/runs/9004907852/job/24739004289?pr=2183#step:6:799

Some of the convergence tests are showing big red flags. I don't think it's just a tolerance thing.

[ChrisRackauckas]

Is more logic for step rejections required? I.e. can you use the convergence rate of a rejected step?

[oscardssmith]

The DelayDiffEq failures seem real (although the tests that are failing do seem pretty bad)