Specify loss function directly for NeuralPDE?

[raminammour]

(Originally posted on discourse)

In NeuralPDE you normally define the PDE ( rhs = lhs) symbolically and the loss function is built internally, loosely \int dvars ||lhs(vars) - rhs(vars)||^2.

Is there a way to specify the objective function symbolically directly? Let’s say you are crazy enough and just want to optimize
\int dvars cos(lhs(vars) + sin(rhs(vars)) . What would be the easiest way to do that?

@ChrisRackauckas directed me to open an issue to maybe add some support/API to that without having to hack too much code.

Cheers!

[zoemcc]

The current best way to do that would be to add your special loss function through the additional_loss function functionality and then pass in a dummy pde equation such as x~x for your pde equation so that it doesn't optimize the original loss function. Or you could leave the original loss function if you want to optimize a sum of your special loss and the generated loss.

Once we make a release for the adaptive loss PR that just merged ( #482 ), you can leave the original pde as-is and then pass in a 0 weight for that particular pde equation using the NonAdaptiveLoss struct

NeuralPDE.jl/src/pinns_pde_solve.jl

Line 255 in d983a0a

mutable struct NonAdaptiveLoss{T <: Real} <: AbstractAdaptiveLoss

. This would be slightly less computationally efficient than the current workaround I suggested since you would compute the generated loss each iteration but is conceptually simpler than the workaround.

Unfortunately this means that you'd have to perform the numerical integral approximations on your side within the additional_loss without using the TrainingStrategy internal machinery that we do use to abstract over the sampling for the integral. Alternatively if you can figure out a transformation of your input equation such that the generated loss function is the one that you want to optimize, that could work (although with your example I don't think it's possible exactly because the loss function has a square and your loss function requires negative values). But if you have a strictly nonnegative loss function you could pass in sqrt(your_loss_function)~0 as a pde equation and the generated loss function should be equivalent to your goal unless I'm making some mental math error.

Going forward I think it could make sense to add another type of additional_loss input that gets integrated over the domain using the internal sampling strategies that the user picks, or perhaps some other way of symbolically transforming the loss function that is generated.

[raminammour]

Thank you for your reply, I will have to dig in deeper to figure out the details but I had figured out that I would need to implement the numerical integral approximation on my side with additional_loss.

My thinking was, since you support integro-differential equations, it should be possible to define the loss function symbolically with using integrals over the domain, and the rest would follow from the internals since you are having to do that work anyway to construct the loss function. I am confident it is possible, not sure how hard it would be to support that API though :)

I had also thought about the appropriate transformations and had come up with similar forms, but I will have to dig deeper to make it technically work (positivity and other constraints, as you said...).

[ChrisRackauckas]

An example of this is shown in the Fokker-Plank documentation. Cheers

[raminammour]

Thanks for the reply and the help. Does this only allow definitions on the discretized solution phi directly?

In the Fokker-Plank example, suppose I wanted to impose $\frac{\partial p(x)}{\partial x} =1$ as additional loss, can I define this symbolically (Dx(p(x)) -1) or would I still have to approximate the derivative numerically myself inside the definition of the additional loss function?

[ChrisRackauckas]

The docs show how to do it yourself, but for this case you can note that boundary conditions essentially are just constraints, and you can use the integro-differential equation support to just define a constraint that the integral over the whole domain is equal to one. That is equivalent to then what is done "by hand" in the example.