Update Violation Energy for Projected Gradient

[LucasBoTang]

In the current implementation of GradientProjection, the constraints violation energy (modules/solvers.py#L45) is calculated once. Given that the energy remains constant during the gradient descent, its gradients will also be constant. Therefore, there's no need to update solution values in an iterative manner.

However, according to the DC3 paper Sec 3.2, it is not very clear but implies the operation for each gradient step that also recalculates the violation energy based on the new solution values.

Given this implication, our approach may need revision.

[drgona]

@LucasBoTang thanks for the feedback.
As far as I remember, the code of the DC3 paper also used multiple steps with the same gradient computation. This is a tradeoff between speed and accuracy. What we could do is implement both versions and add an input argument to switch between these two implementations.

[drgona]

I suggest moving this discussion to the discussion/ideas channel, where we can develop the new feature request before PR.

[LucasBoTang]

Thank you @drgona ! Let's discuss on ideas channel.

However, I noticed that the DC3 official code method.py#L299 calculates the gradient with a new value Y_new per iteration. The method ineq_partial_grad, found here, is used to derive a fresh gradient at every step rather than reusing the same one.

[LucasBoTang]

I developed a new request #110.

[drgona]

This feature was merged into the develop branch and will be deployed to master.