WIP: Adaptive Scalar Basis such as Ranl

[cortner]

This PR will finally implement scalar bases of the form

   R_{k} = sum_q U_{kq} P_q

where k is a general multi-index. Construction and evaluation is working so far. Before merging, we need more tests and maybe a few examples how to initialize the coefficients, e.g. a diagonal, and Michele's method could be implemented fairly easily.

[cortner]

this is going reasonably well so far - main technical challenge is to clean up the 1p basis because bases of the kind Ranl can be very large. This is related to #71, and might even lead to a resolution of this, and possibly we are then in a position to properly sparsify.

[cortner]

the sparsification looks like it is working ok now; in particular this should now close #71 . The implementation is maybe not the most flexible, but it should be more than sufficient for now.

[cortner]

@MatthiasSachs I'll soon be ready to merge this, and this is the last thing we need to explore sparsification with ACE if there is still interest. I think David and Cas can provide some nice training sets if we want.

[davkovacs]

I have some training sets where NequIP is significantly better than linear ACE (factor of 2) so we could try it there.

[cortner]

start with a large basis and then see if sparsification gets us into nequip territory, yes why not. The idea was to explore different sparsification mechanisms and see which are most "productive" both from the perspective of cost and quality of generalization. Nothing deep but a useful study I think.

[davkovacs]

Is it something you would like to try now before merging it, or is it okay in a few days time / next week?

[cortner]

I'm more thinking of this as a separate project / paper.

[cortner]

btw, @davkovacs the other thing in this PR is the ability to have radial bases with parameters that could in principle be optimized.

[cortner]

TODO

• basis label
• diag coeffs
• test it as part of a product basis
• allow a non-trivial mapping between input and basis, e.g. norm for rr argument

[cortner]

@andresrossb What would be involved in optimizing the XScal1pBasis parameters as part of the parameter estimation? If I can supply rrules, would it be fairly straightforward?

[andresrossb]

@andresrossb What would be involved in optimizing the XScal1pBasis parameters as part of the parameter estimation? If I can supply rrules, would it be fairly straightforward?

One would still need to figure out the parameter wrangling. I assume we could build a structure around it like we did for Linear_ACE, however we would need to think of separating these two layers and then using Chain and functor to get the parameters.
We could also try to add this parameters as another variable in the Linear_ACE layer. For example Dense layers have weights and biases, so we could enhance our current ace layer to have weights for the basis parameters, and another entry for the parameters of XScal1pBasis. If we went this route we might get away with only needing a get_params and set_params for XScal1pBasis and integrating the rrules into the ones that already exist.

[davkovacs]

I'm more thinking of this as a separate project / paper.

Okay, let me know when you are ready to try it and we can discuss what the first test case could be.

[cortner]

@andresrossb so not at all trivial. Realistically it can’t be a chain but must be a graph. Can this be done easily?

I've been seeing this more and more clearly actually that we need another major rewrite of ACE.jl to leverage this structure. Blech. If you have suggestions where to start? But I think this is a new issue.

[MatthiasSachs]

@MatthiasSachs I'll soon be ready to merge this, and this is the last thing we need to explore sparsification with ACE if there is still interest. I think David and Cas can provide some nice training sets if we want.

yes, I would be interested! This may proves also very useful for tensor-valued ACE. Would you be free some time later this week, e.g. Thursday or Friday?

[cortner]

finally ready to merge.

[cortner]

This is now tagged as 0.12.31