How to use torchdyn to solve second-order ODE?

[mach512]

Hi, I am relatively new to using machine learning to solve ODE.

Can you kindly help me understand how to implement second-order ODE with torchdyn?

[Zymrael]

Hi, we have a full tutorial notebook showing you how to implement and solve second-order (and even higher, such as tenth-order) Neural ODEs. Check that out then let us know if you have further, more specific questions :)

[mach512]

Hi, Thank you for your reply. I checked the tutorial but the ODE function wasn't defined in any function. How do we solve the differential equation without defining?

Kindly let me know. Thank you.

[massastrello]

Hi,
In the first example of the tutorial (the 2nd order model), we define the ODE by specifying the neural network f representing its vector field

 f = nn.Sequential(nn.Linear(4, 64), nn.Tanh(), nn.Linear(64, 2))

and subsequently passing it to the NeuralDE class, effectively defining the ODE.

 model = NeuralDE(f, order=2, solver='dopri5',  sensitivity='adjoint')

[mach512]

Hi, I understand that the NN is defined and passed as argument to the class NeuralDE and solving used ode solver. If I am not wrong, it seems more like solving neural network with ODE solver, rather than solving differential equation with neural network. Isn't it correct? Because, I could not see the step where the ODE is defined and the neural network being trained to solve it.

Kindly advise. Thank you

[Zymrael]

To add on what @massastrello said, if you wish to specify an ODE that does not involve neural networks, (or does, but only partially) you can do so by building a custom nn.Module.

You're correct in stating that the standard Neural ODEs framework does not involve solving ODEs with neural networks. The ODE is solved with standard solvers, and the neural network simply parametrizes the vector field and is trained to perform different tasks.

For more on using neural networks to enhance ODE solvers you might be interested in our recent work on Hypersolvers: paper, repo

[mach512]

Hi, Kindly excuse me if I am not asking right questions because I have absolutely no idea of how to use neural networks to solve ODE's. My ODE equation is d2x/dt2=(-2xmbmdx/dt)-(km^2x)+Gmkm(Pm+C*(2theta1)/1+np.exp(theta2(theta3-x))), with boundary conditions f(0)=0.3, f(1)=0.3. And I was thinking of using torchdyn to solve it. Does torchdyn allow to define ODE and solve it by minimizing cost function or some other way? Thank you.

[Zymrael]

What you're describing is a boundary value problem (BVP), which torchdyn (or any Neural ODE suite in Python) does not currently support. In the hypersolver implementation above we also consider IVPs only.

[mach512]

Hi, Okay. Doesn't torchdiffeq support solving differential equations by minimizing the cost function through neural networks? I am not clear if I have presented my problem correctly.

Thank you.

[Zymrael]

torchdiffeq solves ODEs (vector field parametrized by neural networks) with classical numerical schemes, in order to perform tasks such as image classification. It does not train neural networks to solve ODEs via cost function minimization :)

[mach512]

Hi. Okay. I didn't know that. But, a general question. Let's say, you are solving ODE's (I don't know what's vector parametrized means?) through torchdyn. How is it possible to define a neural network and solve it by using solver without any equation? I am failing to see the equation here, since ODE's means differential equation. Numerically or analytically, when we solve, we define and implement solvers to solve them. But here, there is no equation being defined and the neural network is being solved through solvers, which I find slightly confusing. After getting results, how do you validate your result? Do you compare with numerical simulation or analytical simulation to check if the result is correct?

[Zymrael]

The ODE here is defined as the forward pass of the neural network passed to NeuralDE. In the simplest form (fully connected neural network) it's a series of nonlinear matrix multiplications interleaved with element-wise nonlinearities. This is what the solver is solving as the ODE.

[Zymrael]

Closing the Issue for the moment.