Symbolic save_idxs first pass

[xtalax]

No description provided.

[YingboMa]

Should save_idxs be handled in the solver or DiffEqBase?

[xtalax]

save_idxs is currently handled in the solver with the integrator, this is just translating symbolic indexes to integer to use that machinery, which I think makes most sense to do at problem construction time as this is the symbolic/"classic" boundary. This will also need to be handled in SciMLBase to make observed variables work again with the differing sized solution, and to fix symbolic indexing of the solution.

Do you mean that this should be a kwarg to solve rather than ODEProblem?

[xtalax]

This is working, with the other Base PR - this is the simplest change that works, but I am open to other suggestions.

The trouble is getting the observed functions to work when they are constructed to expect all states at prob construction time, if we want to do this at solve time. This would mean allocating a dummy array that is larger than the solution to call the observed functions with, which does rather defeat the purpose of this keyword argument

@ChrisRackauckas see this for answer to your question, how to avoid dummy arrays?

[ChrisRackauckas]

It would be a breaking change to move this to the problem instead of solve. Also, I don't get why it "should" be here. It makes it hard to re-solve and just save something else. Why should it be in the problem than in the solve?

[xtalax]

see above for my concerns

[ChrisRackauckas]

The trouble is getting the observed functions to work when they are constructed to expect all states at prob construction time, if we want to do this at solve time. This would mean allocating a dummy array that is larger than the solution to call the observed functions with, which does rather defeat the purpose of this keyword argument

If you do it in the solver though, how are you doing the interpolations?

[xtalax]

I thought that the interpolations were over time rather than idx, so we'd be able to use the same setup. When are the interpolations constructed?

[ChrisRackauckas]

They are constructed during the solve, but only for the states. So if you're planning to extend the u, how are you planning to extend the k?

[YingboMa]

Only solvers like BDF can handle that, because the underlaying interpolant doesn't depend on f at all.

[xtalax]

They are constructed during the solve, but only for the states. So if you're planning to extend the u, how are you planning to extend the k?

My apologies, I'm not familiar with the shorthand k, what are you referring to?

[YingboMa]

ks are the stage derivatives in the Runge-Kutta setting https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#Explicit_Runge%E2%80%93Kutta_methods

They are required to form the dense output. See section 4 of https://www.sciencedirect.com/science/article/pii/S0898122111004706

[xtalax]

The states are not in general linked in any kind of spatial configuration during an ODE solve, and appear only to be linked by the calculation of the truncation error in that paper.

I am unsure whether it is possible to ignore the contribution of the unsaved states to the truncation error, but regardless this is the only thing that might be changed by changing the saved states, unless I'm missing something.

Meaning I don't think that the calculation of the k would change, except from possibly dropping unnecessary states from the calculation

[ChrisRackauckas]

But if you make save_idxs have an observable, how do you extend k? That's something we haven't worked out yet.

[xtalax]

save_idxs having an observable is at the moment just automatically saving its state dependencies as a first pass

[xtalax]

superseded by #2071