Image Denoising with Generative Adversarial Network
Branch: master
Clone or download
manumathewthomas
manumathewthomas final change
final change
Latest commit 4f1484e May 3, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
libs Final Changes May 2, 2017
static Final Changes May 2, 2017
templates Final Changes May 2, 2017
Project Proposal with summary of papers reviewed.pdf final change May 2, 2017
README.md Update README.md May 2, 2017
conv_helper.py Final Changes May 2, 2017
main.py Final Changes May 2, 2017
model.py Final Changes May 2, 2017
paper.pdf Paper May 2, 2017
result1.PNG results. May 2, 2017
result2.png results. May 2, 2017
result3.PNG results. May 2, 2017
test.py Final Changes May 2, 2017
train.py Final Changes May 2, 2017
utils.py Final Changes May 2, 2017

README.md

Denoising with GAN

Paper | Video

Introduction

Animation movie companies like Pixar and Dreamworks render their 3d scenes using a technique called Pathtracing which enables them to create high quality photorealistic frames. Pathtracing involves shooting 1000’s of rays into a pixel randomly(Monte Carlo) which will then hit the objects in the scene and based on the reflective property of the object the rays reflect or refract or get absorbed. The colors returned by these rays are averaged to get the color of the pixel and this process is repeated for all the pixels. Due to the computational complexity it might take 8-16 hours to render a single frame.

We are proposing a neural network based solution for reducing 8-16 hours to a couple of seconds using a Generative Adversarial Network. The main idea behind this proposed method is to render using small number of samples per pixel (let say 4 spp or 8 spp instead of 32K spp) and pass the noisy image to our network, which will generate a photorealistic image with high quality.

Demo Video Link

IMAGE ALT TEXT HERE

Table of Contents

Installation

To run the project you will need:

Running

Once you have all the depenedencies ready, do the folowing:

Download the dataset extract it to a folder named 'dataset' (ONLY if you want to train, not needed to run).

Extract the CKPT files to a folder named 'Checkpoints'

Run main.py -- python3 main.py

Go to the browser, if you are running it on a server then [ip-address]:8888, if you are on your local machine then localhost:8888

Dataset

We picked random 40 images from pixar movies, added gaussian noise of different standard deviation, 5 sets of 5 different standard deviation making a total of 1000 images for the training set. For validation we used 10 images completely different from the training set and added gaussian noise. For testing we had both added gaussian images and real noisy images.

Hyperparameters

  • Number of iterations - 10K
  • Adversarial Loss Factor - 0.5
  • Pixel Loss Factor - 1.0
  • Feature Loss Factor - 1.0
  • Smoothness Loss Factor - 0.0001

Results

3D rendering test data: alt text

Real noise images: alt text

CT-Scan: alt text

Improvements

  • Increase the num of iteration to 100K.
  • Train the network for different noises.
  • Make it work on a real-time app.

Credits