update StiffDDE benchmarks

[ChrisRackauckas]

No description provided.

[ChrisRackauckas]

These ones show the same regression @devmotion @YingboMa @chriselrod

[devmotion]

Is it possible to view the regressions somewhere? I checked the other PR but it was not clear to me how to figure out what the differences are and what package versions are used by the existing benchmarks (it seems there was no Manifest.toml?).

[ChrisRackauckas]

It's going to be hard because the old benchmarks were just "it's on v1.4 sometime" 🤷 . This now has a manifest and will have a history of all manifests, but yes, tracking down what it is might be hard. It might just be easier to simply dig in and find a 30%-50% speedup, which is around the amount that regressed.

[devmotion]

These are the changes since 5.24.0 (which was set in the Project.toml): SciML/DelayDiffEq.jl@v5.24.0...v5.30.1

It contains mainly:

• updates of CI
• switch from Roots to NonlinearSolve (Use NonlinearSolve instead Roots DelayDiffEq.jl#197 and Fix type inference DelayDiffEq.jl#200)
• separate propagated time stops and user-provided time stops (Separate propagated time stops and discontinuities DelayDiffEq.jl#196)
• some updates to the types to keep up with OrdinaryDiffEq (e.g., Sync DelayDiffEq with changes in OrdinaryDiffEq DelayDiffEq.jl#193, allow skipping error checks DelayDiffEq.jl#207 and Update for stiffness detection anti-looping DelayDiffEq.jl#211)
• some other minor fixes that e.g. fix autoswitch (fix autoswitch caching DelayDiffEq.jl#189), simplify StochasticDelayDiffEq (More general history function DelayDiffEq.jl#201 and Generalize Discontinuity DelayDiffEq.jl#202), fix saveat (Enforce correct saveat after final tstop change DelayDiffEq.jl#206), fix reset of saveiter_dense (Fix reset of saveiter_dense in reinit! DelayDiffEq.jl#210), and fix callback initialization (Fix initialization of callbacks DelayDiffEq.jl#191)

[devmotion]

I wonder if NonlinearSolve causes the regressions?

[chriselrod]

Roots.jl's performance is pretty bad, so I would've expected NonlinearSolve to help.

[ChrisRackauckas]

It could just be some broadcast regressions, if broadcast is used in the repo, that needs to get replaced to FastBroadcast.jl

[chriselrod]

Unless it's using @. somewhere instead of @.., it should already be updated if using an updated DiffEqBase:
https://github.com/SciML/DelayDiffEq.jl/blob/17c4cddabf043afec9346500a75c03a40af68049/src/DelayDiffEq.jl#L17

But it could be that it didn't fix the problem in DelayDiffEq (for both NonStiff and Stiff), in which case we'd need to look into it and fix it.

All these SciMLBenchmarks being run now are the first set of real performance tests for FastBroadcast.jl, so not unlikely that we run into something to fix.

I'll change how safeivdep works to make it possible for others to whitelist their own types, so that the various array types used in the SciML & DiffEq ecosystems can opt in.
I'm also fine with making it just always ivdep like the old one did, apparently without causing problems.

[ChrisRackauckas]

I'm just going to merge the DelayDiffEq ones, but we should keep this in mind. We can track it in SciML/DelayDiffEq.jl#213