Expand limited PDE forms

[znicolaou]

In building the PDELibrary, the library is constructed by taking combinations of the derivatives and the library_functions applied to all permutations of dependent (field) variables. There are two significant limitations:

1. The library does not include the possibility of nonlinearity on the derivatives. This limits the class of PDE's that can be discovered to only (special kinds of) quasilinear systems, but not all pdes can be put in this form. An option to apply library_functions to derivative terms would address this.

2. The library_functions are assumed to act dependent variables only, but this means the PDE cannot have any explicit dependence on the independent variables (i.e., only constant-coefficient equations can be discovered). This excludes important cases with externally-imposed heterogeneity, such as external potentials in the Schrodinger equation, and cases where the dependent variables are not all spatial, such as the p.grad_x term in the Botzmann equation. The form of the arguments to the library_functions could be changed to address this. For example, lambda x,u where x is a list of dependent variables and u a list of independent variables.

[akaptano]

I'm putting in new features for 1) and 2). Actually 2) is already half-implemented on the new branch through the SpatiotemporalLibrary, which allows for a library of the independent variables (the spatial and temporal coordinates, for instance). See the bottom of the Example 1 Jupiter notebook. Feel free to use this but it is liable to significant changes in the next few weeks. I say it is only half implemented because I need to show how this can be tensor-produced with other libraries to create terms that are mixed independent-dependent variables, such as the p grad(x) term you mentioned.

More updates to come soon

[akaptano]

I should also mention it is hard to say what is the best balance of complexity/flexibility for these sorts of questions.

For instance, for a PDE that governs a vector q(x, t), you can actually just pass [q, x, t] as the variable "x" in model.fit, and the library functions will be applied to all those variables (although you need to reformat x, t = np.meshgrid(x, t, indexing='ij' to get them in the same shape as q), allowing you to do the option 2) you mentioned.

The question is if this "hack" is sufficient, because it has the downside of being unintuitive, too much work for the user-end of the code, and grows very large because all the possible combinatoric products of terms in the library are generated.

[znicolaou]

Got it, thanks!

I see the hack is an option now. Note though that the functional dependence on independent and dependent variables is often expected to be very different, so applying the same library_functions may give too many library terms in some cases without further constraints. For parametrically-driven waves, for example, we may want to include sinusoidal temporal dependence, but limit to only linear and cubic dependence on field variables.

[akaptano]

Agreed, I think this is the best reason to put in a separate library, and then allow for tensor products. This should be available with SpatiotemporalLibrary soon.

[akaptano]

Frankly, this the same reason why we often want a separate library for the derivative terms. I think the most general solution would just be allow for separate libraries for all the possible features in the data, so for an input [q, q_x, q_xx, x, ...] with unique libraries [theta_q, theta_{q_x}, ...] it would concatenate and take tensor products of whatever subset of these libraries you want. This would allow for the most functionality but I'll need to think about the best way to implement that avoids a lot of extra work on the user front end.

[znicolaou]

Makes sense! Happy to brainstorm if you want to bounce any ideas. I may start playing with code and learning how to contribute soon too.

[akaptano]

Hey @znicolaou,

I have implemented a GeneralizedLibrary class that does everything we want. See the updated Example 1 on the PDEFIND branch and let me know if you have any questions. To respond to your recent email, heads up we implement Sam Rudy's STRidge algorithm 2 (which it looks like is what you used to get the nice results) in Example 10 so feel free to copy and paste that over wherever. This is a good idea to scan the hyper parameters and look for the lowest error (in Rudy's case, track the MSE on the test set). Feel free to close this issue when/if you are satisfied with the fix.

Best,
Alan

[akaptano]

Oh also @znicolaou and @briandesilva I also implemented a class TensoredLibrary that takes two libraries and tensors them together, so that you can use syntax like PolynomialLibrary() * FourierLibrary() for the tensored library just like PolynomialLibrary() + FourierLibrary() gives you the concatenated library. This was required anyways for GeneralizedLibrary. Note that you can tensor as many libraries together as you want with GeneralizedLibrary.

[znicolaou]

Thanks @akaptano! Will give it a spin soon :)

[akaptano]

Oh also, I removed SpatiotemporalLibrary since this is now can be done straightforwardly and with more precision with the GeneralizedLibrary. SpatiotemporalLibrary was just a CustomLibrary applied to the spatiotemporal variables.

[akaptano]

It occurs to me that we also don't need the SINDyPILibrary anymore, although it might be good to keep since using the GeneralizedLibrary is a bit more advanced.

[akaptano]

Update, the new release is out (everything is merged into the master branch), so closing this for now. Feel free to reopen if any issues are encountered.