Migrate functionality to HighDimPDE.jl

[ChrisRackauckas]

• NNPDENS
• NNPDEHan
• NNPDEHan_tests.jl
• NNPDENS_tests.jl
• Solving a 100-dimensional Hamilton-Jacobi-Bellman Equation tutorial
• https://neuralpde.sciml.ai/dev/examples/blackscholes/ tutorial
• https://neuralpde.sciml.ai/dev/solvers/deep_fbsde/

[vboussange]

The migration of NNPDENS and the associated test file is addressed in the specific issue SciML/HighDimPDE.jl#30.

[vboussange]

Work in progress on branch https://github.com/SciML/HighDimPDE.jl/tree/NeuralPDE_integration.

Thanks to @ashutosh-b-b, the integration is almost completed and partially works. A major change is that now in HighDimPDE.jl, only one algorithm, DeepBSDE, replaces the former NNPDENS and NNPDEHAN from NeuralPDE.jl. The latter were basically the same algorithms, except that NNPDENS was used to call a solve function that would allow the use of SDE solvers from DifferentialEquations.jl to solve the reformulated stochastic problem. What happens now is that if an sdealg is provided in the solve function, then a different solve function which formulates the SDE problem is called (see : https://github.com/SciML/HighDimPDE.jl/blob/a83198b733f94865824f7d6dfa29188077ccebd5/src/DeepBSDE.jl#L72 ).

What still needs to be done:

• Clean up the tests, located in test/DeepBSDE_Han.jl and test/DeepBSDE_Han.jl. Style should be similar to e.g. test/DeepSplitting.jl.
• Fix the tests that are not working (marked by a #TODO comment).
• Add a page on the documentation under solver algorithms, documenting DeepBSDE.
• Migrate the tutorials listed above.

[ChrisRackauckas]

@ashutosh-b-b you had a PR around this? What's left here?

[ashutosh-b-b]

Its merged. HighDimPDE#32, I guess mostly the documentation is left there.

[vboussange]

@ChrisRackauckas,@ashutosh-b-b, the PR is merged, but on the branch https://github.com/SciML/HighDimPDE.jl/tree/NeuralPDE_integration and not on the main branch, as stated above.

@ashutosh-b-b, Could you take look at the test files DeepBSDE_Han.jl and DeepBSDE.jl, and fix the tests with annotated #TODO? On my side, I will complete the documentation, and migrate the tutorials.

[ashutosh-b-b]

@ChrisRackauckas @vboussange the problem with some of the tests is ambiguity regarding methods on Tracked variables. For example:
MethodError: vcat(::TrackedArray{…,Vector{Float32}}, ::Vector{Tracker.TrackedReal{Float32}}) is ambiguous

is what we get for DeepBSDE - Nonlinear Black-Scholes Equation with Default Risk

and

MethodError: no method matching Float32(::Tracker.TrackedReal{Float32}) for
DeepBSDE - Black-Scholes-Barenblatt equation

[ChrisRackauckas]

If you end up with Array{TrackedReal} you'll take quite a performance hit. Are you using any mutation that would trigger this?

[ashutosh-b-b]

Yeap, its here. and here

[ChrisRackauckas]

That's not mutation, but would still trigger it. Maybe do Tracker.collect(vcat(σ(X,_p,t),_σᵀ∇u))?

[vboussange]

@ChrisRackauckas @vboussange the problem with some of the tests is ambiguity regarding methods on Tracked variables. For example: MethodError: vcat(::TrackedArray{…,Vector{Float32}}, ::Vector{Tracker.TrackedReal{Float32}}) is ambiguous

is what we get for DeepBSDE - Nonlinear Black-Scholes Equation with Default Risk

and

MethodError: no method matching Float32(::Tracker.TrackedReal{Float32}) for DeepBSDE - Black-Scholes-Barenblatt equation

@ashutosh-b-b @ChrisRackauckas , where are we now with this? Would be nice to finally merge all of this work and make a release, since NeuralPDE.jl does not provide anymore the algorithms. Why do we use Tracker here? Wouldn't Zygote be more appropriate?