Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Leaner serialization for solution types #629

Closed
5 tasks done
ChrisRackauckas opened this issue Feb 22, 2024 · 18 comments
Closed
5 tasks done

Leaner serialization for solution types #629

ChrisRackauckas opened this issue Feb 22, 2024 · 18 comments
Assignees

Comments

@ChrisRackauckas
Copy link
Member

ChrisRackauckas commented Feb 22, 2024

The solution types contain functions, and things like JLD2 don't like functions. So it would be nice to have a "strip_solution` function that generates a much leaner form of the solution for simply doing easier serialization. Here's how that should look:

  1. we should make a new trait for has_lazy_interpolation in https://github.com/SciML/SciMLBase.jl/blob/master/src/alg_traits.jl which is then set to true on BS5 and VernX solvers. strip_solution should then give an informative error message that these problems are not supported by this function. Just some proper error handling, and with that out of the way the other solvers should be free to go.
  2. https://github.com/SciML/SciMLBase.jl/blob/master/src/solutions/ode_solutions.jl#L102-L118 we can strip out any of the function information from this object. I.e. make a new problem where prob and alg are nothing.
  3. Downstream, sol.interp holds some extra information as well. So in SciMLBase we need strip_interp(::AbstractInterpolation). The SciMLBase ones are just the identity function, so that's easy https://github.com/SciML/SciMLBase.jl/blob/master/src/interpolation.jl .
  4. At this point any non-OrdinaryDiffEq algorithm will strip well. The only downstream interp to handle is the OrdinaryDiffEq one. https://github.com/SciML/OrdinaryDiffEq.jl/blob/master/src/interp_func.jl#L4-L15 . f is only in there for the lazy interpolations, so set it to nothing. You need to keep the caches because that's used for dispatch, but at this point you're done with explicit methods.
  5. For implicit methods, you need to strip https://github.com/SciML/OrdinaryDiffEq.jl/blob/master/src/caches/rosenbrock_caches.jl#L30-L33 jac_config and grad_config, as those contain function information. As a simple thing, you can just create a new cache with everything nothing though since this is only used for dispatch (and addsteps!, but since you know these caches are non-lazy the interpolation addsteps! post solution is trivial so it won't use anything in the cache). We need to make good functionality for this anyways for better default solvers (CC @oscardssmith), so it might be a good time to do this now.

With that done strip_solution(sol) should return a lean solution with no function information in it (or rather, any function information in it is in SciMLBase, which also defines the structure so that's a required import) and so it should work just fine with any BSON, JLD2, etc. package.

@jClugstor
Copy link
Contributor

For the first item, should it be has_lazy_interpolation(alg::AbstractDEAlgorithm) = false,? Then e.g. has_lazy_interpolation(alg::Vern6) = true, would that go somewhere in here : https://github.com/SciML/OrdinaryDiffEq.jl/tree/07470f95ab5d76864e2d1565468305459bf39b00/lib/OrdinaryDiffEqVerner/src ?

@ChrisRackauckas
Copy link
Member Author

yes, and traits go in alg_utils.jl, you'll see one in that folder.

@ChrisRackauckas
Copy link
Member Author

It's just Verner and BS5 (low order RK)

@jClugstor
Copy link
Contributor

Where should the strip functions go? SciMLBase utils.jl ?

@ChrisRackauckas
Copy link
Member Author

BTW I created the trait SciML/OrdinaryDiffEq.jl@9734f49 since I ended up needing it. It needs to get raised to the SciMLBase level.

Where should the strip functions go? SciMLBase utils.jl ?

yes that's fine.

@jClugstor
Copy link
Contributor

jClugstor commented Aug 7, 2024

This adds a has_lazy_interpolation trait to SciMLBase, and sets strip_interpolation for the SciMLBase interpolations to identity, since they don't have functions in them.
#761

This adds strip_interpolation and strip_solution functions for the stuff in OrdinaryDiffEq. They basically copy the objects, just setting the function field to nothing.
SciML/OrdinaryDiffEq.jl#2316

Now I need to make a strip_solution for SciMLSolution, and work on the implicit method stuff.

@oscardssmith oscardssmith self-assigned this Aug 8, 2024
@ChrisRackauckas
Copy link
Member Author

The latter

@jClugstor
Copy link
Contributor

It looks like SciML/OrdinaryDiffEq.jl#2320 will combine the cache types, so I'll only need to deal with RosenbrockConstantCache and RosenbrockCache, right?

@jClugstor
Copy link
Contributor

Also, is there any way to tell whether an algorithm is implicit/has caches, other than looking at the docs? Would that be worth having as a trait?

@oscardssmith
Copy link
Contributor

It looks like SciML/OrdinaryDiffEq.jl#2320 will combine the cache types, so I'll only need to deal with RosenbrockConstantCache and RosenbrockCache, right?

right

@ChrisRackauckas
Copy link
Member Author

@jClugstor what's the next step here?

@jClugstor
Copy link
Contributor

Figuring out how to handle the explicit and implicit solvers seperately, since the Rosenbrock caches have some function information. It looks like there are imminent changes to the Rosenbrock caches so I was planning on waiting on those before trying to implement the stripping.

@ChrisRackauckas
Copy link
Member Author

Alright, but do the other methods first?

@ChrisRackauckas
Copy link
Member Author

@jClugstor are you adding integration tests today?

@jClugstor
Copy link
Contributor

I was just going to ask about this.

  • Should strip_solution be exported ?
  • What kind of integration tests do I need to add? A test to check that serialization with JLD2 and BSON works after strip_solution ?

@ChrisRackauckas
Copy link
Member Author

  1. Probably not.
  2. Downstream, just show that when you strip the interpolation with OrdinaryDiffEq that you have no functions inside of there. Can be in OrdinaryDiffEq.

@ChrisRackauckas
Copy link
Member Author

Just needs docs. @jClugstor add to the solution interface page in DiffEqDocs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants