[BUG] Error in CG implementation

[HuyD24]

betty/betty/hypergradient/cg.py

Line 53 in 39457e1

p_new = [rr + beta * pp for rr, pp in zip(r, p)]

I think p_new should be calculated based on r_new instead of r here. In the Wikipedia version (https://en.wikipedia.org/wiki/Conjugate_gradient_method) the corresponding formula is $p_{k+1} = r_{k+1} + \beta_k p_k$ with $r_{k+1}$ corresponding to r_new.

[sangkeun00]

Hello @HuyD24,

Thanks for your interest and pointing out the error in my code. I think you are right, and thus I fixed the code to use r_new instead of r. When I ran several benchmarks with both r and r_new, I didn't experience any noticeable performance differences, and hope your experiments also didn't get affected by this bug. Let me know if you have other concerns.

Best,
Sang

[sangkeun00]

Closing the issue as the bug is fixed.