Adaptive learning rate in NetKet

[wlwuphy]

Some papers on solving many-body quantum systems using AI/ML techniques impose constraint on the Fubini-Study distance between the wave functions of two adjacent iterations. To put it simply, the learning rate is adjusted so that large changes of the wave function are prevented and the iteration process is more stable. However, I see that netket.optimizer.Sgd only accepts constant learning rate. I wonder what changes need to be done if I want to implement adaptive learning rate? Will it be troublesome and involve a lot of works? Thanks!

[PhilipVinc]

The algorithm they use is Stochastic reconfiguration https://netket.readthedocs.io/en/latest/api/_generated/driver/netket.driver.VMC_SR.html#netket.driver.VMC_SR

[wlwuphy]

Yes, but in my understanding SR only uses the preconditioner, namely updating the parameters as $$\theta_i\rightarrow \theta_i-\eta S^{-1}g_i$$ instead of $$\theta_i\rightarrow \theta_i-\eta g_i$$. However, the original SR does not include adaptive learning rate $$\eta$$. By adaptive learning rate, I mean changing $$\eta$$ so that

Eq.14 is satisfied (This is taken from https://doi.org/10.1103/PhysRevC.107.034320). I am wondering what does it take to implement this in NetKet.