Variable boundary conditions

[chang-change]

Dear lulu,DeepXDE is a great tool.However, I have some questions. I am solving a 2-dimensional heat equation, and I want to use spatial coordinates and boundary conditions as input to the neural network, and the output is the temperature field of the solution domain. My goal is to train with different boundary conditions and then input a new boundary condition to the trained neural network so that the trained neural network can directly input a temperature field. Can DeepXDE do this? Thank you very much!

[praksharma]

Hi, I also use DeepXDE. I guess you can do this with parametrization. You need to heavily tweak the source code.
But if you think this from a physics perspective. well-posed 2D heat equation has a unique solution. You are asking a neural network to throw multiple unique solutions for each well-posed problem for a different BC vector. This would require a huge amount of data and I would suggest you make your problem underdetermined i.e. put parameters more than the number of samples.

From my experience of solving heat equations, especially steady-state. Be very careful with your BCs. If you have two adjacent boundaries with conflicting BCs, then you are literally introducing a discontinuity. Well, Nvidia Modulus uses signed distance function to kick those points out of the training process.

For example, I solved steady state 3D heat conduction problem with different temperature at each face using Nvidia modulus. It took me 120 layers and 150 neuron in each layer. Basically, an underdetermined system (assume we remove the PDE loss) for my training set of 20k points. Of course, DeepXDE fails on most of the 3D problems with conflicting BCs.

[lululxvi]

More comments:

• There are several discussions on parametric PDE. Check FAQ "Q: Solve parametric PDEs.", but I haven't listed all the related issues, and you can also search in the issues.
• Depending on exactly what you try to do, you may need or not to tweak the code.
• For the conflicting adjacent BCs, the current version of DeepXDE should remove all the points of those corners. @praksharma Do you have a specific example (2D/3D), where DeepXDE didn't remove the corner points?

[praksharma]

@lululxvi Thanks for the instant reply. I was trying to see whether PINNs can solve 2D and 3D problems with conflicting adjacent BCs. Of course, those baseline PINNs, learning rate annealing with encoder, Neural Tangent kernel were designed for this.

Surprisingly, 2D heat conduction with conflicting adjacent BCs did work with DeepXDE. Here are the results in one of the closed issues on this repo.

When I tried the 3D heat conduction, things became very weird with DeepXDE. I applied different temperatures on each face.

This is the FEM solution.

This is the solution after 60k Epochs, black points are those out of bounds i.e. [100,200]

Here is the loss curve for individual faces, PDE and total loss

I also tried putting weight coefficient but the result is same. I tried to train the same on 40 hidden layers with 150 Neurons in each layers.

Here is the loss function:

And the result is even worse.

Here is the link to the Jupyter notebook: https://github.com/praksharma/temp/blob/main/2D%20heat%20conduction.ipynb
Please don't mind the title in the first cell, it is wrong.

[chang-change]

@lululxvi @praksharma Thanks your instant response. I have another question. For Conflicting Adjacent Dirichlet boundary conditions, can a hard constraint method be used? A typical case is the Lid-Driven cavity. Where, the velocity of the upper boundary is a constant, while the velocity of the other three boundaries is 0. By modifying the network output, I use the function:xy(1-x)*(1-y)*N+f(x,y). But I don't know how to design f(x,y) to satisfy these boundary conditions. I do it according to the method provided by the paper 'Artificial Neural Networks for Solving Ordinary and Partial Differential Equations'.

[praksharma]

@1445590317 Do you mean you want to hard constrain the corner with conflicting BC?
I don't know if this is possible because @lululxvi mentioned in this comment that the current version of DeepXDE will remove those points. So, you will end hard constraining those points that does not even exist in the training dataset.

Also, if you read DeepXDE paper, page no. 6, second paragraph. They suggest using soft-constraint for stiff-PDEs. In convection dominant flows we add little bit of artificial diffusion to smooth the discontinuity.

Still, you can try it for yourself. Hope this link will help you.

[lululxvi]

@praksharma If it works well for 2D, then ideally it would work for 3D. Could you try to scale the solution by 100, i.e., instead of using 100 and 200 for the BC, using 1 and 2?

@1445590317 A hard constraint BC could be used, but I am not sure about the performance. To implement a hard constraint BC, you don't have to consider all BCs --- you can use hard constraints for some BC (e.g., upper boundary), and for other BCs, you can still use soft ones.

[chang-change]

@lululxvi @praksharma thanks your response. I will try it. And I get inspiration from the papers, 'Learning the solution operator of parametric partial differential equations with physics-informed DeepOnet' and 'Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators', to solve parametric PDE. The DeepONet is magical.
Another question is that from the paper, 'Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations', we know that the author propose two models (continuous time and discrete time models) to solve PDE. A lot of examples from the DeepXDE doc are continuous time, discrete time models(Runge–Kutta methods with q stages) can be used in DeepXDE? Thank you!

[praksharma]

@chang-change Well, the discrete-time model from the original paper was hardly investigated in any other paper. This paper which involves a fusion of finite difference and PINN is much better. At least you don't need to formulate anything that is problem specific. They have solved stiff-PDEs in 2D and look promising. Nevertheless, can't say anything about the authenticity as the preprint was uploaded on 3rd March.
HFD-PINN: http://arxiv.org/abs/2202.07926
My only concern is that they also use the signed distance function similar to Nvidia Modulus by having an idea of how the solution will look like which is completely nonsensical.

[chang-change]

@praksharma Thanks for your response. The idea that combines finite difference and PINN looks good.

[praksharma]

@chang-change yes interesting but the problem is scalability. A couple of weeks ago Nvidia was presenting their PINN library, which they call Nvidia Modulus in my research group.

I have to say their library is the state of art PINN and you will only learn those PINNs that are scalable to complicated 3D problems. There are 100s of PINN models, but very few are scalable to real problems.

Nvidia has spent 3 years, first Simnet then Modulus. Their library is very comprehensive. Below is a list of PINN models that they have fully discovered.

DeepXDE is great and very cheap compared to Nvidia Modulus. Also, the Nvidia modulus uses very cheap tricks to make things work (signed distance function), but it works.

As a student, I am open to any library that helps me in my work. Nvidia Modulus being proprietary and open source is great. Hopefully, we can see the implementation of multiple GPUs in DeepXDE with horovod. Also, I would like to appreciate @lululxvi for being so active in solving the issues.

[lululxvi]

@chang-change Yes, DeepONet, including physics-informed DeepONet, could be a better choice for parametric PDE. There are many recent papers on DeepONet, and you can find some at https://deepxde.readthedocs.io/en/latest/user/research.html#deeponet-mionet-deepm-mnet

In terms of discrete-time models or finite difference + PINN:

• Discrete-time models are not supported in DeepXDE, partially due to the reasons mentioned by @praksharma
• Also, I am not convinced yet if the finite difference and PINN is a good future direction.

[lululxvi]

Nvidia has spent 3 years, first Simnet then Modulus. Their library is very comprehensive.

DeepXDE is great and very cheap compared to Nvidia Modulus. Also, the Nvidia modulus uses very cheap tricks to make things work (signed distance function), but it works.

Hopefully, we can see the implementation of multiple GPUs in DeepXDE with horovod. Also, I would like to appreciate @lululxvi for being so active in solving the issues.

I read the source code of Simnet (but not Modulus yet), and yes, compared to Modulus, DeepXDE is designed to be light and cheap.

Yes, the support of multiple GPUs via Horovod is in the TO-DO list of DeepXDE. Nvidia has a whole team developing Modulus, but for DeepXDE, we have much fewer resources, so it is difficult to guarantee when Horovod will be supported.

@praksharma May I invite you to join the slack of DeepXDE? If you would like to, please send me an email for the slack invitation.