Could not find the generator loss.

[PeterouZh]

Hi,

thanks for your great job.

When I read the code, I found there is only the discriminator loss and no generator loss. In other words, there is no adversarial training in MEALv2, which is different from my intuition. I want to know what is the advantage of just using the discriminator.

[szq0214]

Hi @PeterouZh, generator loss is the similarity loss in our framework to produce the same distribution as teachers', i.e., the KL divergence loss. Conventional adversarial training uses alternate updating but since the input and output of our discriminator and generator (student) are differentiable (not images), we can train the whole pipeline jointly.

[PeterouZh]

Thanks for your reply. It's very interesting. Maybe a detailed ablation study is needed to verify the effectiveness of the discriminator loss.