Call to solve fails with `LoadError: "No matching function wrapper was found"

[ConnectedSystems]

This is with DifferentialEquations v7.2, Julia v1.8 and v1.7.3 on Windows 11.

Trying to replicate with a simpler example, will update once I do.

Output from status -m attached

DiffEq_7.2_JLv1.8_W11.txt

[ConnectedSystems]

MWE as promised:

using DifferentialEquations
using Plots


f(u, p, t) = 1.01 * u
u0 = 1 / 2
tspan = (0.0, 1.0)

# This works without issue
prob = ODEProblem(f, u0, tspan)

# This also works
prob = ODEProblem{false,true}(f, u0, tspan)

# These error with "No matching function wrapper was found"
# prob = ODEProblem{true,false}(f, u0, tspan)   # this one is not relevant to the example, but was how things were for our ODE
# prob = ODEProblem{false,false}(f, u0, tspan)

solver::RK4 = RK4()
sol = solve(prob, solver, reltol = 1e-8, abstol = 1e-9)


plot(sol, linewidth = 5, title = "Solution to the linear ODE with a thick line",
    xaxis = "Time (t)", yaxis = "u(t) (in μm)", label = "My Thick Line!") # legend=false
plot!(sol.t, t -> 0.5 * exp(1.01t), lw = 3, ls = :dash, label = "True Solution!")

Can't remember what the second boolean in ODEProblem{false,false} related to and the docs don't seem to mention it at all now. I take it this is now removed?

[ChrisRackauckas]

ODEProblem{true,false}(f, u0, tspan) will of course not work because there is no in-place function given, and you cannot define in-place for scalars. So that's expected, and will throw a special error message saying this, so please let me know if that isn't clear.

prob = ODEProblem{false,false}(f, u0, tspan) you can force the FunctionWrappers but they are only designed to work by default on Vector{Float64}, so forcing it here will fail. And yes, it should use the SciMLBase.FunctionWrapperSpecialize etc. stuff.

Can't remember what the second boolean in ODEProblem{false,false} related to and the docs don't seem to mention it at all now. I take it this is now removed?

It was never documented. Using undocumented features should be done with some caution 😅. The behavior changed and was allowed to change because it was unreleased not public API. What exactly it does will be explained in more detail when it's released. What you really want here is ODEProblem{false,SciMLBase.NoSpecialize}, but until it's released I would not rely on that.

[ChrisRackauckas]

So okay, this looks fine since it matches the expected behavior of the specializations.

[ConnectedSystems]

ODEProblem{true,false}(f, u0, tspan) will of course not work because there is no in-place function given, and you cannot define in-place for scalars.

Just to clarify, I only included it in the MWE to indicate that it does not work for the ODE function we're using, whereas previously it did.
As I noted with a comment:

# this one is not relevant to the example, but was how things were for our ODE

It was never documented.

Oh really? I don't know how I came across it then because I've never dug too deeply into the package codebase.
I don't recall why it was needed in the first place, but okay, thanks for the info!

[ChrisRackauckas]

If you find it somewhere in the docs, let me know 😅

I only included it in the MWE to indicate that it does not work for the ODE function we're using, whereas previously it did.

Yes, because previously false meant what is now NoSpecialize, but it's instead given the fallback now of FunctionWrapperSpecialize. But since it's now a multi-valued logic, the boolean handling is just deprecated. Hopefully others aren't using it because I didn't plan to have it documented until it fully solved recompilation 😅

[asprionj]

Sorry to open this up again. We just ran into the same problem. We used to construct the ODEProblem(...) without the parameters, everything worked fine, all the time. Now since an up of the packages, things just don't work anymore. It seems that the change from the default "no specicalization" to "FunctionWrapperSpecialize" breaks it. If I manually set it to SciMLBase.NoSpecialize, everything works fine again.

So if I have to set this parameter to make it work for my way of formulating the model function, it has to be documented. What's the basic idea behind it? When (and why) does the "automatic specialization" not work (such as in our case)?

[ChrisRackauckas]

So if I have to set this parameter to make it work for my way of formulating the model function, it has to be documented.

It is documented in detail in the docstrings.

https://docs.sciml.ai/DiffEqDocs/stable/types/ode_types/#SciMLBase.ODEProblem
https://docs.sciml.ai/SciMLBase/stable/interfaces/Problems/#Specialization-Levels

What's the basic idea behind it?

https://sciml.ai/news/2022/09/21/compile_time/#handling_higher_order_functions_controlling_specialization

When (and why) does the "automatic specialization" not work (such as in our case)?

It should work now? If it doesn't, that's worth an issue. The case of this issue, at least the MWE that I've seen, is fine in the latest versions.

[asprionj]

Thanks for the links to the docs, it's clear now what it's all about. Our use-case at hand involves generating ODEProblems for use in optimisation loops, so I guess FullSpecialize would be the best choice here.

We're not on DiffEq 7.6.0, and we do have the issue of the "No matching function wrapper was found" Error. I'd have to extract some MWE reproducing our situation, but am currently vey short on time. I'll see if I can manage to still do it.

[ChrisRackauckas]

Since the system is relatively new, v7.4, 7.5, and 7.6 all had some stuff that was ironing out bugs. I wouldn't even try to look for an MWE before updating as it may already be fixed.

Our use-case at hand involves generating ODEProblems for use in optimisation loops, so I guess FullSpecialize would be the best choice here.

Possibly. Though the late specialization form in one of the later DiffEq versions fixed a lot of issues here.

[asprionj]

Haha, nasty typo. I did mean "We're now on DiffEq 7.6.0...".

[ChrisRackauckas]

Okay, well then an MWE would be helpful to figure out what's going on here. But FullSpecialize is all that's needed to get full runtime performance without this issue.

[asprionj]

Could not fully pinpoint the issue. But it seems to be caused by specifying the initial state as a function. In the original formulation, I use (p,t)->model_x0(p) to set the initial state. If I call model_x0 on the parameters before and then use the resulting vector in the construction of ODEProblem, the error is gone. (But we do need this "dynamic" setting of initial state as a function of the parameters.)

[ChrisRackauckas]

oh interesting. Yeah that case may need a few more tests. Would be good to get an issue and an MWE.

[asprionj]

Done: #916