conditional implementation

[PieraRiccio]

Hi, did you also implement a conditional implementation, so that I can pass files with labels, as in the original TensorFlow implementation?

[lucidrains]

@PieraRiccio Hey Piera! I haven't implemented the conditional part yet

I would have to think about how this could work as a CLI before moving forward. Perhaps I could ask for the labels as some standard data format that matches the filename of the images, and then have an extra flag for passing in the labels file

[PieraRiccio]

@lucidrains Hi. :) Thank you for your answer. I think that a usual modality to do that is to retrieve the labels from subfolders of the main images folder. What do you think about this?

In order to run conditional training in the official TF implementation, I used the code shared in this issue, maybe it can be helpful:
NVlabs/stylegan2-ada#22

[lucidrains]

Sure, that works! How do you have your files -> label organized at the moment?

[PieraRiccio]

I have a main folder and 4 sub-folders (which correspond to the 4 labels) :)

[lucidrains]

@PieraRiccio oh that may work! let me think about it before executing on the code

[lucidrains]

@PieraRiccio so I looked over the official code, and it seems like labels are consumed by the generator and discriminator in the following way

in both G and D, the label id is first projected to the feature map size and normalized

in the generator, it is concatenated to the latent vector before being passed through the mapping network

in the discriminator, it seems to be also passed through a separate mapping network https://github.com/NVlabs/stylegan2-ada/blob/main/training/networks.py#L520, but the number of layers is defaulted to 0, so I'm not sure if it is being used. in the end, it is multiplied with the image embedding just before the output https://github.com/NVlabs/stylegan2-ada/blob/main/training/networks.py#L619

[matthewchung74]

@lucidrains cc @PieraRiccio I have a very dirty attempt at the conditional logic https://github.com/foobar8675/stylegan2-pytorch/tree/dev, running on cifar100 right now.

[lucidrains]

@foobar8675 Amazing Matthew! 💯 💯

[PieraRiccio]

@foobar8675 Thank you! :)

[matthewchung74]

I've been trying to run on cifar100, which was probably not realistic (only 600 images / class) . starting again with cifar10 like the nvidia guys did. not sure how much it matters, but am running with these flags.

python -m stylegan2_pytorch.cli --results_dir /tmp --models_dir /tmp 
--aug-prob 0.25 
--aug-types [translation,cutout,color] 
--dataset-aug-prob .5  
--trunc-psi 0.75 
--top-k-training 
--attn-layers [1,2] 
--batch-size 16
--data ../datasets/cifar10/train/ --name cifar10 

step 8884, with 2 rows manually annotated:

(48 hrs later) i stopped the training after 4 days. it didn't seem to be any better than what was posted above.

[PieraRiccio]

@foobar8675 Given the small number of images for each label, have you also tried using adaptive discriminator augmentation? :)

[matthewchung74]

@PieraRiccio thanks for the suggestion. I have not, but I do have augmentation set to .25. My understanding, which might be wrong, is that ADA will kick in with augmentation if it senses augmentation is needed and otherwise not augment. since I have augmentation always set, I'm not sure if it would help?

However, I might be misunderstanding. Do you have any thoughts?

[BrentOGara]

@matthewchung74 This: #194 (comment) looks very cool, exactly what I need... but the link no longer works. Is this still available anywhere? Thanks.

[matthewchung74]

@BrentOGara unfortunately, i deleted that repo, since it wasn't quite as good as what I wanted. if you are looking for a conditional stylegan2, nvidia's implementation supports it. https://github.com/NVlabs/stylegan2-ada-pytorch

[BrentOGara]

@matthewchung74 Thank you for the link, I'll see if I can figure it out. I'm still very new to this field.