Why lambda_min not used in Backward?

[luoyetx]

As the title described. In forward part, lambda won't be smaller than lambda_min, but in backward part, it doesn't compare with lambda_min. Is there a reason to do so or just a mistake ?

[wy1iu]

There is no need to compare lambda in back prop. If lambda reaches lambda_min, it will stay unchanged during the rest of the optimization.

[luoyetx]

then, I think this line and this line won't promise. Since iter_ is always increased by 1 during forward, lambda will be smaller than lambda_min. Right?

[wy1iu]

Yes, you are right about this. My apologies. It seems like I commit an old version of my code. Without lambda_min, it is okay for MNIST and CIFAR10, which is the reason I did not find out this mistake. But in the implementation for LFW and CIFAR100, these lines has been deleted. You will need lambda_min to reproduce performance in LFW and CIFAR100. Really sorry about that. I will fix this bug now.

[wy1iu]

Thanks very much for reporting the bug. Really appreciate it. :)