Glead loss computed more than necessary in Dreg phase

[Alaa-Arbi]

In loss_glead.py in line 207. if do_Dmain or do_Dr1:. My understanding is that this if block is supposed to compute the GAN loss for the real images + the glead regularization loss and or the R1 regularization loss. And I guess that if only R1 regularization is wanted (meaning do_Dmain is False then neither the gan loss nor the glead regularization loss is supposed to be computed. While in fact the GAN loss is not computed (line 231 if do_Dmain: ), the glead loss is still computed one extra time (one more time than for fake images). So my question is whether this was or wasn't intentional.

[EzioBy]

Hi. Firstly, I think the glead loss in D_reg phase is not duplicated to the one in D_main phase because the discriminator weights would be updated after D_main phase. And incorporating the glead loss in D_reg phase may alleviate the influence of R1 penalty on the glead strategy. Yet I did not conduct systematic ablation studies on this.