Generative Adversarial Networks are state-of-the-art deep models that learn to produce realistic synthtic images as a byproduct of an adversarial game between two competing learning agents, called a generator and a discriminator.
In order to understand more about how GANs learn to represent class information internally, we need a way to visualize the activations of the layers inside the network. I have assembled a dimensionality reduction algorithm which combines PCA and t-SNE in order to represent the multi-dimensional activations in a humanly understandable 2-dimensional space.
The process is to first train a GAN and a classifier on the MNIST dataset, and then generate new images while recording the activations inside the GAN. We predict the labels of the generated images, and then use them to color the activations of the intermediate layers when plotting them. This allows interesting observations to be made about the appearance and propagation of class information inside of the GAN.
This repository is a section of the work completed in partial fulfilment of the requirements for the degree of BSc (Hons) in Computer Science at the University of Manchester. The dissertation details the conclusions of the research Andrei_Iliescu_Report.pdf.
- checkpoint/ Pre-trained GAN and classifier models.
- train/gan/ Generated images over every epoch of training.
- visualizations/ Saved visualizations of intermediate GAN layers.
- Andrei_Iliescu_Report.pdf Dissertation introducing GANs and detailing the visualizations performed here.
The model has been run with GPU-accelerated TensorFlow using the following packages:
- Python 3.5
- Keras 2.2
- Tensorflow 1.12
- numpy 1.15
- scikit-learn 0.2
- matplotlib 2.2
-
Make sure all dependencies are installed.
-
Clone the repository.
-
First, train the GAN model (skip this step if you want to use the pre-trained model).
python3 train_gan.py- Then, train the classifier model (skip this step if you want to use the pre-trained model).
python3 train_classifier.py- Finally, produce visualizations of the internal GAN layers.
python3 visualize_gan.pyGAN-generated MNIST over 1, 20 and 40 epochs.
Visualization of the 3rd discriminator layer on real images.
Visualization of the 3rd generator layer.
Visualization of the 3rd discriminator layer on generated images.
Everything is licensed under the MIT license.
[1] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y. Generative Adversarial Networks. In Advances in neural information processing systems, pp. 2672–2680, 2014a.
[2] van der Maaten, L., Hinton, G. Visualizing Data using t-SNE. Journal of Machine Learning Research 9 (2008) 2579-2605.Aniruddha Ghosh and Tony Veale. 2016.
[3] Radford, A., Metz, L., Chintala, S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. ICLR 2016.