Conditional WGAN

[aosokin]

Hi, have you tried to apply WGAN for conditional image generation?
Say, in the simplest scenario of conditioning on the class label.
I'm trying to do that, but observe some weird behavior:

1. If I add an extra head for the class label (like AC-GAN), the WGAN head just wins and another one is simply ignored. This is understandable because its gradients saturate, but the ones of WGAN do not.
2. If I do like CGAN, i.e. feed the critic the class label as well, the discriminator loss does not really move the right direction.

Any suggestions?

[rafaelvalle]

Have you tried passing your noise through a dense layer and then concatenating its output with the labels.

[aosokin]

@rafaelvalle Yes, I've tried combining CGAN or ACGAN, but it never worked.
I always saw one of the two failure modes: Wassestein objective converges, but conditioning is just ignored; Wasserstein objective does not converge at all.

[rafaelvalle]

@aosokin Did you try providing both the critic and generator with the condition?
There's some paper, I don't remember which, that suggests using conditioning whenever possible.

[LukasMosser]

@rafaelvalle there should be some theoretical reasoning behind this. I would assume that adding the conditioning vector once to the data should be sufficient. I believe that is the case for vanilla GANs.

[aosokin]

@rafaelvalle yes, I've tried several things like they go in the ACGAN paper, but never could make it work.

[rafaelvalle]

@aosokin Did you try looking at the gradients close to your noise and condition vector?
Are the images normalized ?
How many classes do you have and what type of data?

[aosokin]

@rafaelvalle

Did you try looking at the gradients close to your noise and condition vector?

Towards convergence most of the parameters are clipped, so the gradients all look outside.

Are the images normalized ?

yes, to [-1, 1]

How many classes do you have and what type of data?

I've tried 6 classes on some images. I should probably try it on a MNIST, but did not have time.

I'm curious if there is any success story on conditioning WGAN on anything.

[rafaelvalle]

@aosokin I was successful conditioning WGAN on ~70 classes with chromatic images.
I've read that conditional image generation with GANs, not WGANs, benefit from L2 loss. Maybe there's an adaptation go WGAN?
https://arxiv.org/pdf/1611.07004v1.pdf

[aosokin]

@rafaelvalle

I was successful conditioning WGAN on ~70 classes with chromatic images.

Could you share a code snippet on what worked?
Would be very helpful.

I've read that conditional image generation with GANs, not WGANs, benefit from L2 loss.

But that's image conditioned on image, which is even harder. For me, even conditioning on labels did not work. Maybe I just did something wrong.

[rafaelvalle]

@aosokin
The code for running the GAN is the first link and the models the second. Let me know if you have any questions or figure out your problem!

https://github.com/rafaelvalle/neural_network_control_improvisation/blob/master/wcgan_text.py
https://github.com/rafaelvalle/neural_network_control_improvisation/blob/master/models.py

[aosokin]

@rafaelvalle Great, thanks a lot! I'll try this version out

[rafaelvalle]

@aosokin
Have you tried implementing things from zi2zi: Master Chinese Calligraphy with Conditional Adversarial Networks.

@martinarjovsky any suggestions on how to combine the WGAN loss with a classification loss, say categorical crossentropy?

[aosokin]

@rafaelvalle Hi, I've tried some things along the lines of your code, but it still does not work, but I still observe the same result: the conditioning is ignored in the end of the day.
The key difference to what I as doing before appeared in when to merge image and label signal in the critic: I was doing this early at the image level, you are merging it in later. I've tried both and see no difference.

zi2zi: Master Chinese Calligraphy with Conditional Adversarial Networks.

This seems to do a very different thing.

Have you tried passing your noise through a dense layer and then concatenating its output with the labels.

Yes, I've tried that, but it did not change the outcome at all.

[rafaelvalle]

@aosokin Did you try conditioning à la wavenet, z = tanh(Wk * X + VkC), where * is convolution and W and V are the weights at layer K?

[tkoch96]

@rafaelvalle Hi, trying to view your first link above but it doesn't seem to be working. Could you re-post?

[omair-kg]

Hi @aosokin , I'm trying something similar with conditional WGAN and observed similar behaviour, either conditioning is completely ignored or the network doesn't converge.

I added a second loss to the generator as well (L2 loss) to 'guide' the results in the correct direction. This setup is pretty sensitive to lambda (the multiplier) for the GAN loss. The training now appears to be headed in the right direction but it is still early days yet.

[appierys]

I've successfully trained a conditional WGAN as well. But I am using WGAN-GP version. Maybe that's the difference?
BTW, I found that I need to keep training even after original WGAN loss is converged.

[aosokin]

@appierys Hi, thanks for the notice. Could you share some code snippets on how you did this?

[shaform]

I've spent some time to implement WACGAN-GP for MNIST this morning. The results seem to be acceptable. The sample jupyter notebook is as follows: WACGAN-GP-MNIST

[weihancug]

@appierys Hi,Could you share some code to show how you do it

[appierys]

@weihancug Hi, I think the notebook provided above already shows a working conditional WGAN ?
WAGCAN-GP-MNIST

[shaform]

I've found other implementations of cWGAN:

• https://github.com/fairytale0011/Conditional-WassersteinGAN
• https://github.com/vsooda/mxnet-wgan

Looks like WGAN is ok with conditional settings since many people could reproduce the results.

[kongyanye]

Here's an implementation of conditional WGAN-GP. https://github.com/kongyanye/cwgan-gp