Image Denoising with Generative Adversarial Network
Branch: master
Clone or download
manumathewthomas final change
final change
Latest commit 4f1484e May 3, 2017
Type Name Latest commit message Commit time
Failed to load latest commit information.
libs Final Changes May 2, 2017
static Final Changes May 2, 2017
templates Final Changes May 2, 2017
Project Proposal with summary of papers reviewed.pdf final change May 2, 2017 Update May 2, 2017 Final Changes May 2, 2017 Final Changes May 2, 2017 Final Changes May 2, 2017
paper.pdf Paper May 2, 2017
result1.PNG results. May 2, 2017
result2.png results. May 2, 2017
result3.PNG results. May 2, 2017 Final Changes May 2, 2017 Final Changes May 2, 2017 Final Changes May 2, 2017

Denoising with GAN

Paper | Video


Animation movie companies like Pixar and Dreamworks render their 3d scenes using a technique called Pathtracing which enables them to create high quality photorealistic frames. Pathtracing involves shooting 1000’s of rays into a pixel randomly(Monte Carlo) which will then hit the objects in the scene and based on the reflective property of the object the rays reflect or refract or get absorbed. The colors returned by these rays are averaged to get the color of the pixel and this process is repeated for all the pixels. Due to the computational complexity it might take 8-16 hours to render a single frame.

We are proposing a neural network based solution for reducing 8-16 hours to a couple of seconds using a Generative Adversarial Network. The main idea behind this proposed method is to render using small number of samples per pixel (let say 4 spp or 8 spp instead of 32K spp) and pass the noisy image to our network, which will generate a photorealistic image with high quality.

Demo Video Link


Table of Contents


To run the project you will need:


Once you have all the depenedencies ready, do the folowing:

Download the dataset extract it to a folder named 'dataset' (ONLY if you want to train, not needed to run).

Extract the CKPT files to a folder named 'Checkpoints'

Run -- python3

Go to the browser, if you are running it on a server then [ip-address]:8888, if you are on your local machine then localhost:8888


We picked random 40 images from pixar movies, added gaussian noise of different standard deviation, 5 sets of 5 different standard deviation making a total of 1000 images for the training set. For validation we used 10 images completely different from the training set and added gaussian noise. For testing we had both added gaussian images and real noisy images.


  • Number of iterations - 10K
  • Adversarial Loss Factor - 0.5
  • Pixel Loss Factor - 1.0
  • Feature Loss Factor - 1.0
  • Smoothness Loss Factor - 0.0001


3D rendering test data: alt text

Real noise images: alt text

CT-Scan: alt text


  • Increase the num of iteration to 100K.
  • Train the network for different noises.
  • Make it work on a real-time app.