A question on the computation of Hessian-vector product

[YiifeiWang]

In the function dataloader_hv_product() under the class hessian(), in line 86-87, it follows
'''
THv = [torch.randn(p.size()).to(device) for p in self.params
] # accumulate result
'''
I am wondering why it uses random initialization instead of zero initialization. (Although in actual computation, with large data number, this initialization is approximate to zero.)

[htwang14]

Thanks a lot to the authors for releasing these codes of their excellent work. But I do agree with @YiifeiWang that it is more appropriate to use zero initialization here. In my own experiments, when using random initialization, the power iteration converges poorly and the returned top eigenvalues vary a lot among different runs. Please point out if my understanding is wrong.

[yaozhewei]

Thanks for pointing this out. This is a mistake when we clean up the code. It is fixed now.