different update frequencies of different modules

[Honlan]

Hi~ thanks a lot for your awesome work!

I find that during each iteration, D is optimized for twice:

d_loss, d_losses_latent = compute_d_loss(
    nets, args, x_real, y_org, y_trg, z_trg=z_trg, masks=masks)
self._reset_grad()
d_loss.backward()
optims.discriminator.step()

d_loss, d_losses_ref = compute_d_loss(
    nets, args, x_real, y_org, y_trg, x_ref=x_ref, masks=masks)
self._reset_grad()
d_loss.backward()
optims.discriminator.step()

G is also optimized for twice:

g_loss, g_losses_latent = compute_g_loss(
    nets, args, x_real, y_org, y_trg, z_trgs=[z_trg, z_trg2], masks=masks)
self._reset_grad()
g_loss.backward()
optims.generator.step()
optims.mapping_network.step()
optims.style_encoder.step()

g_loss, g_losses_ref = compute_g_loss(
    nets, args, x_real, y_org, y_trg, x_refs=[x_ref, x_ref2], masks=masks)
self._reset_grad()
g_loss.backward()
optims.generator.step()

However, mapping_network and style_encoder are only optimized for once.

Could you explain it for me? many thanks again.

[yunjey]

@Honlan The first phase is for latent-guided synthesis and the second phase is for reference-guided synthesis. The only difference is whether to use the mapping network or the style encoder to extract target style codes.

[Honlan]

So, why mapping_network and style_encoder are optimized during the first phase (optims.mapping_network.step(), optims.style_encoder.step()), but these two modules are not optimized during the second phase?