Advanced Deep Learning with Keras

This is the code repository for Advanced Deep Learning with Keras, published by Packt. It contains all the supporting project files necessary to work through the book from start to finish.

About the Book

This book covers advanced deep learning techniques to create successful AI. Using MLPs, CNNs, and RNNs as building blocks to more advanced techniques, you’ll study deep neural network architectures, Autoencoders, Generative Adversarial Networks (GANs), Variational AutoEncoders (VAEs), and Deep Reinforcement Learning (DRL) critical to many cutting-edge AI results.

Instructions and Navigation

All of the code is organized into folders. Each folder starts with a number followed by the application name. For example, chapter2-deep-networks.

The code will look like the following:

def encoder_layer(inputs,
                  filters=16,
                  kernel_size=3,
                  strides=2,
                  activation='relu',
                  instance_norm=True):
"""Builds a generic encoder layer made of Conv2D-IN-LeakyReLU 
IN is optional, LeakyReLU may be replaced by ReLU
"""
conv = Conv2D(filters=filters,
              kernel_size=kernel_size,
              strides=strides,
              padding='same')

Related Products

• Deep Reinforcement Learning Hands-On

• Deep Learning with Keras

• Reinforcement Learning with TensorFlow

Advanced Deep Learning with Keras sample code used in the book

Chapter 1 - Introduction

1. MLP on MNIST
2. CNN on MNIST
3. RNN on MNIST

Chapter 2 - Deep Networks

1. Functional API on MNIST
2. Y-Network on MNIST
3. ResNet v1 and v2 on CIFAR10
4. DenseNet on CIFAR10

Chapter 3 - AutoEncoders

1. Denoising AutoEncoders

Sample outputs for random digits:

2. Colorization AutoEncoder

Sample outputs for random cifar10 images:

Chapter 4 - Generative Adversarial Network (GAN)

1. Deep Convolutional GAN (DCGAN)

Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015).

Sample outputs for random digits:

2. Conditional (GAN)

Mirza, Mehdi, and Simon Osindero. "Conditional generative adversarial nets." arXiv preprint arXiv:1411.1784 (2014).

Sample outputs for digits 0 to 9:

Chapter 5 - Improved GAN

1. Wasserstein GAN (WGAN)

Arjovsky, Martin, Soumith Chintala, and Léon Bottou. "Wasserstein GAN." arXiv preprint arXiv:1701.07875 (2017).

Sample outputs for random digits:

2. Least Squares GAN (LSGAN)

Mao, Xudong, et al. "Least squares generative adversarial networks." 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.

Sample outputs for random digits:

3. Auxiliary Classfier GAN (ACGAN)

Odena, Augustus, Christopher Olah, and Jonathon Shlens. "Conditional image synthesis with auxiliary classifier GANs. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, PMLR 70, 2017."

Sample outputs for digits 0 to 9:

Chapter 6 - GAN with Disentangled Latent Representations

1. Information Maximizing GAN (InfoGAN)

Chen, Xi, et al. "Infogan: Interpretable representation learning by information maximizing generative adversarial nets." Advances in Neural Information Processing Systems. 2016.

Sample outputs for digits 0 to 9:

2. Stacked GAN

Huang, Xun, et al. "Stacked generative adversarial networks." IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Vol. 2. 2017

Sample outputs for digits 0 to 9:

Chapter 7 - Cross-Domain GAN

1. CycleGAN

Zhu, Jun-Yan, et al. "Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks." 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017.

Sample outputs for random cifar10 images:

Sample outputs for MNIST to SVHN:

Chapter 8 - Variational Autoencoders (VAE)

1. VAE MLP MNIST
2. VAE CNN MNIST
3. Conditional VAE and Beta VAE

Kingma, Diederik P., and Max Welling. "Auto-encoding Variational Bayes." arXiv preprint arXiv:1312.6114 (2013).

Sohn, Kihyuk, Honglak Lee, and Xinchen Yan. "Learning structured output representation using deep conditional generative models." Advances in Neural Information Processing Systems. 2015.

I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. β-VAE: Learning basic visual concepts with a constrained variational framework. ICLR, 2017.

Generated MNIST by navigating the latent space:

Chapter 9 - Deep Reinforcement Learning

1. Q-Learning
2. Q-Learning on Frozen Lake Environment
3. DQN and DDQN on Cartpole Environment

Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529

DQN on Cartpole Environment:

Chapter 10 - Policy Gradient Methods

1. REINFORCE, REINFORCE with Baseline, Actor-Critic, A2C

Sutton and Barto, Reinforcement Learning: An Introduction

Mnih, Volodymyr, et al. "Asynchronous methods for deep reinforcement learning." International conference on machine learning. 2016.

Policy Gradient on MountainCar Continuous Environment:

Citation

If you find this work useful, please cite:

@book{atienza2018advanced,
  title={Advanced Deep Learning with Keras},
  author={Atienza, Rowel},
  year={2018},
  publisher={Packt Publishing Ltd}
}