Better BPINN Log Posterior formulation and Experiments

[AstitvaAggarwal]

Checklist

• Appropriate tests were added
• Any code changes were done in a way that does not break public API
• All documentation related to code changes were updated
• The new code follows the
  contributor guidelines, in particular the SciML Style Guide and
  COLPRAC.
• Any new documentation only uses public API

Additional context

Uses extra loss term

[AstitvaAggarwal]

@ChrisRackauckas, @Vaibhavdixit02 this has code for both PDE and ODE case over Master Branch.

[ChrisRackauckas]

What do you mean?

There's some conflicts here.

[AstitvaAggarwal]

the extra loss funciton in case of the ODE solver and PDE solver, im still working on the PDE solver case. Ill fix the conflicts as im currently focusing on this pr

[Vaibhavdixit02]

Type it or create with comprehension

[Vaibhavdixit02]

same

[Vaibhavdixit02]

I think broadly this looks good already. Do you have any results for comparison of the new likelihood?

[ChrisRackauckas]

Agreed I look at this and it seems ready-ish to merge. What's the status here?

[AstitvaAggarwal]

Im still running experiments, I've managed to get one test set pass for the new implementation but that's a simple example, the Lorentz equations example fails as I still need to tune the training process better. Overall im working on the new test problems

[Vaibhavdixit02]

Okay sounds good. What do you mean by failing for the Lorenz one, how does it fail? Also Lorenz is way too hard to be the test problem, use the LV instead

[AstitvaAggarwal]

I mean the estimated parameters are not within tolerated error bounds, ive not been able to get HMC or NUTS reduce the Hamiltonian energy enough in all the runs till now. Yeah, will try the same for KS and LV now

[ChrisRackauckas]

You may want to rebase to master so it should go faster.

[AstitvaAggarwal]

@ChrisRackauckas i apologise for the delayed PR, ive been testing the implementation thoroughly hence the delay(single solve takes 1.5-2hrs). ive shared results uptil now with @Vaibhavdixit02. The results are positive but not significant enough, this is for the LV case as the PDE residual loss is misleading (allows zero gradients trivial solution for inverse problem setting). this PR involves the new loss implementation, some tests, and a correction to the existing PDE BPINN solver(was hidden in the details of the Domain point generation function present in the normal PINN solver). I will add more tests in a seperate PR.

[AstitvaAggarwal]

For your reference:
green and pink curve for new implementation(sol5_2), blue and brown for old implementation(sol3)