(This project is a part of my Summer Internship 2020 from Makers Fablab)
The Pix2Pix Generative Adversarial Network, or GAN, is an approach to training a deep convolutional neural network for image-to-image translation tasks.
The careful configuration of architecture as a type of image-conditional GAN allows for both the generation of large images compared to prior GAN models (e.g. such as 256×256 pixels) and the capability of performing well on a variety of different image-to- image translation tasks.
Pix2Pix is a Generative Adversarial Network, or GAN, model designed for general purpose image-to-image translation.
The approach was presented by Phillip Isola, et al. in their 2016 paper titled “Image-to- Image Translation with Conditional Adversarial Networks” and presented at CVPR in 2017.
https://phillipi.github.io/pix2pix/
The GAN architecture is comprised of a generator model for outputting new plausible synthetic images, and a discriminator model that classifies images as real (from the dataset) or fake (generated). The discriminator model is updated directly, whereas the generator model is updated via the discriminator model. As such, the two models are trained simultaneously in an adversarial process where the generator seeks to better fool the discriminator and the discriminator seeks to better identify the counterfeit images.
The Pix2Pix model is a type of conditional GAN, or cGAN, where the generation of the output image is conditional on an input, in this case, a source image. The discriminator is provided both with a source image and the target image and must determine whether the target is a plausible transformation of the source image.
The generator is trained via adversarial loss, which encourages the generator to generate plausible images in the target domain. The generator is also updated via L1 loss measured between the generated image and the expected output image. This additional loss encourages the generator model to create plausible translations of the source image.
The Pix2Pix GAN has been demonstrated on a range of image-to-image translation tasks such as converting maps to satellite photographs, black and white photographs to color, and sketches of products to product photographs.
Now that we are familiar with the Pix2Pix GAN, let’s prepare a dataset that we can use with image-to-image translation.
• Download Maps Dataset (maps.tar.gz) http://efrosgans.eecs.berkeley.edu/pix2pix/datasets/maps.tar.gz
Download the dataset and unzip it into your current working directory. This will create a directory called “maps” with the following structure:
1 maps
2 ├── train
3 └── val
The train folder contains 1,097 images, whereas the validation dataset contains 1,099 images.
Images have a digit filename and are in JPEG format. Each image is 1,200 pixels wide and 600 pixels tall and contains both the satellite image on the left and the Google maps image on the right.
1) The desired Image to Image translation is done from Sattelite data to Google Maps and can be implemented vice-versa.
2) It is also concluded that number of epochs does not make a huge difference in the final result.
• Standalone Satellite. Develop an example of translating standalone Google map images to satellite images, as we did for satellite to Google map images.
• New Image. Locate a satellite image for an entirely new location and translate it to a Google map and consider the result compared to the actual image in Google maps.
• More Training. Continue training the model for another 100 epochs and evaluate whether the additional training results in further improvements in image quality.
• Image Augmentation. Use some minor image augmentation during training as described in the Pix2Pix paper and evaluate whether it results in better quality generated images