DCGAN_Tensorflow

Usage

$ python3 main.py

NOTE :

On colab notebook use the following commmand: !git clone link-to-repo %run main.py

Contributed by: Shivamshree Gupta

References:

Title : Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

Author : Alec Radford, Luke Metz, Soumith Chintala

Link : https://arxiv.org/pdf/1511.06434.pdf

SUMMARY

• DCGAN

Deep Convolutional Generative Adversarial Networks

Introduction

One way to build good natural images is by training Generative Adversarial Networks (GANS). ,and we can later even reuse parts of the generator and discriminator networks as feature extractors for supervised tasks .DCGANs is actually one of the class of GANs using CNNs architecture for both of its components i.e. a generator and a discriminator.

Model Architecture

In general, GANs consist of a generator and a discriminator. These two are separately a CNN architecture and are trained together.

GENERATOR :

A ANN model which is aimed for the generation of new images. They took in input a random noise z and various convolution transpose layers are applied and it generates a image i.e. a matrices of pixel values (G(z)) .Generator never get to see the real world actual images or the training dataset of images.

DISCRIMINATOR:

A CNN classification model used to classify whether an image passed to it is real or fake (generated by the generator).It take image from training examples x and from those generated by generator (G (z)) and predicts the probability of the image to be real (D(x)) or fake (D (G (z))).

Now, while training the model, generator tries to increase the discriminator error as it tries to fool discriminator by improving its generated image so that they resemble real images while discriminator tries to decrease it’s error by trying to judge correctly the real and the fake images. For weights of the model normally initiated ,we first train generator say for y no. of images keeping the discriminator’s weights constant .Then, as generator’s weight are updated ,we train discriminator keeping generator’s weights to be constant for y fake and y real images and this process is then repeated for several epochs using cross entropy loss function.

This becomes a minimax type loss function as above illustrated.

Architecture guidelines for stable Deep Convolutional GANs

• Replace any pooling layers with strided convolutions (discriminator) and fractional-strided convolutions (generator). • Use batchnorm in both the generator and the discriminator except at the generator’s output layer and discriminator’s input layer. • Remove fully connected hidden layers for deeper architectures. • Use ReLU activation in generator for all layers except for the output, which uses Tanh. • Use LeakyReLU activation in the discriminator for all layers.

Implementation and Training details

1. Pre-processing -: scaling images to [-1 1]. No data augmentation.
2. Batch size -: 128.
3. Weights initialization -: using normal distribution.
4. LeakyReLU -: slope of leak=0.2.
5. Adam optimizer-: learning rate=0.0002, momentum=0.5.
6. Epochs -: 50.
7. Noise dimension -: 100

Images generated of MNIST dataset after 50 epochs:

Plot of generator loss vs. discriminator loss

Sources

1. Deep Learning Part -II(CS7015),Lec-1
2. Deep Learning Part -II(CS7015),Lec-2