feat: NonlinearSolveBase + BracketingNonlinearSolve + SimpleNonlinearSolve

[avik-pal]

First step in cutting down on load times. This is mostly independent of #456

1. Move away from DiffEqBase to NonlinearSolveBase in the Simple packages.
2. Split SimpleNonlinearSolve into (SimpleNonlinearSolve has longer load times than Lux which sounds extremely wrong)
   1. SimpleNonlinearSolve -- retains its current functionality but drops DiffEqBase and loads NonlinearSolveBase and BracketingNonlinearSolve
   2. BracketingNonlinearSolve
3. What does NonlinearSolveBase have?
   1. Termination Conditions that were kind of incorrectly put (by me) into DiffEqBase
   2. Forward AD overloads via extensions
   3. Helper functions for norm that is used across NonlinearSolve
   4. Moved the definition of ImmutableNonlinearProblem from SimpleNonlinearSolve
4. What are the other sources of bad load time?
   1. DocStringExtensions -- Remove LibGit2 dep JuliaDocs/DocStringExtensions.jl#167
   2. DifferentiationInterface -- SparseArrays Reduce load time? gdalle/DifferentiationInterface.jl#416
   3. FiniteDiff -- again SparseArrays
   4. MaybeInplace -- again SparseArrays (fixed by refactor: move SparseArrays into an extension MaybeInplace.jl#11)
   5. MLStyle via SciMLBase -- I have been told this is being looked into.
5. Pending downstream changes:
   1. DiffEqBase loads NonlinearSolveBase and adds get_concrete_problem

There's a bunch of stuff in NonlinearSolve that can eventually be moved into base, but that is something to do later on.

Tasklist

• NonlinearSolveBase
  • Base Package
  • Testing
  • ForwardAD Primitives
• BracketingNonlinearSolve
  • All core algorithms
  • Add tests
  • ForwardAD support
• SimpleNonlinearSolve
  • move package over from the other repo
• CI scripts

[avik-pal]

@time begin
    using SimpleNonlinearSolve
    prob_brack = IntervalNonlinearProblem{false}(
        (u, p) -> u^2 - p, (0.0, 25.0), 5)
    solve(prob_brack, ITP())
end
# 1.574422 seconds (1.17 M allocations: 81.038 MiB, 2.57% gc time, 6.58% compilation time)


@time begin
    using BracketingNonlinearSolve, SciMLBase
    prob_brack = IntervalNonlinearProblem{false}(
        (u, p) -> u^2 - p, (0.0, 25.0), 5)
    solve(prob_brack, ITP())
end
# 0.504557 seconds (429.92 k allocations: 30.137 MiB, 6.16% gc time, 25.34% compilation time)

[ChrisRackauckas]

Move away from DiffEqBase to NonlinearSolveBase in the Simple packages.

Should we move the DiffEqBase/solve.jl to SciMLBase? We probably should and make that a bit more generic, and then reuse it in more contexts anyways.

[ChrisRackauckas]

You also want the diffeqbase.jl functionwrappers though.

[avik-pal]

Should we move the DiffEqBase/solve.jl to SciMLBase? We probably should and make that a bit more generic, and then reuse it in more contexts anyways.

I think so. They don't seem to be diffeq specific.

You also want the diffeqbase.jl functionwrappers though.

Not sure what you are referring to

[ChrisRackauckas]

This part: https://github.com/SciML/DiffEqBase.jl/blob/master/src/norecompile.jl