the problem for training loss

[slz929]

Thanks for enlightening work agian.

I train the Mdis method for one source and one target, but I am confused for the loss, and I plot by tensorboard.
And as I think, the adv loss should walk low and the discrimitor loss should walk higher.
but in the loss below, the two losses oscillate around a number. whats wrong with it?

Besides, I infer the training results should be better when training in manner of 1source 1target instead of 1source multi target. But in my training, I dont get good results.

So hope your thought sincerely.

And my training config:
adv loss weight: 0.5
adv learning rate: 1e-5
seg learning rate: 1.25e-5

adversarial loss of one source and one target

dicriminator loss of one source and one target

[SportaXD]

What you observe is a standard behavior of adversarial training. Since the adversarial loss and the discriminator loss have opposite objectives, you shouldn't observe one or the other going down to 0 significantly. Observing this kind of behavior shows that your training is well balanced between discriminator training and adversarial training. Overall, you shouldn't bother too much with these loss values.

On another note, you shouldn't use Mdis for training on one source and one target but a standard AdvEnt for instance. Multi-Dis. is designed to handle more than one target.

Hope it helps.

[slz929]

@SportaXD
Thanks for quick reply.
I am clear.
I just fix the train_mdis fucntion to train with 1source 1target to generate the loss plot above, and not that large improved DA reasults. And in coincidence, it is same with the train_advEnt function to update the loss and etc.