A Caffe Implementation of SRGAN
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
SRGAN update May 14, 2017
include add solver.hpp Oct 26, 2017
pycaffe_srgan add readme.md May 15, 2017
src update May 14, 2017
utils modified the codes and update May 1, 2017
readme.md update notes about the solver.cpp problem Oct 26, 2017

readme.md

Caffe_SRGAN

A caffe implementation of Christian et al's "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network" paper.

Dependencis

Train

  • ubuntu 14.04
  • CUDA 7.5
  • Caffe

Test

1. obtain the SR imgs:

  • ubuntu 14.04
  • pycaffe

2. compute_psnr_ssim:

  • windows 10
  • matlab

Usage

  1. Overload or add the caffe_gan/caffe.proto, solver.cpp and caffe_gan/include/_layer.hpp and caffe_gan/src/_layer.cpp *_layer.cu to your caffe

 cd caffe && make clean && make all && make pycaffe 
 cp -r SRGAN caffe/examples/ 
  1. Preparing training data: randomly crop images of the ImageNet dataset into 216000 sub-imgs and save as .h5 format(my finally training set: 216000x3x74x74, 21600x3x296x296)

    • My implementation is in the win10 matlab : run utils/Generate_data/mygenerate_sr_trainx4.m
  2. Training SRResNet-MSE:

     cd yourpath/caffe && sh examples/SRGAN/train_srres_75s.sh
    
  3. Testing SRResNet-MSE:

    cd yourpath/caffe/examples/SRGAN/ && python srres-deploy.py 
    
  4. Training SRGAN-MSE:

     cd yourpath/caffe && sh examples/SRGAN/train_srgan_is2.sh 
    
  5. Testing SRGAN-MSE:

     cd yourpath/caffe/examples/SRGAN/ && python srres-deploy.py 
    

Benchmarks

After fixing the SRResNet-MSE architecture(Add Scale_layer) and the deploy.prototxt(For test, use global stats in the BN_layer) , the SRResNet-MSE worked well and its results are close to the offical results. My results of SRGAN-MSE are weird, and I don't know what's going wrong with it...

Set5 [4x upscaling]:

Model/Benchmarks PSNR SSIM
SRResNet-MSE(official) 32.05 0.9019
SRResNet-MSE(mine) 32.01 0.8914
SRGAN-MSE(official) 29.40 0.8472
SRGAN-MSE(mine) 32.01 0.8916

Set14 [4x upscaling]:

Model/Benchmarks PSNR SSIM
SRResNet-MSE(official) 28.49 0.8184
SRResNet-MSE(mine) 28.55 0.7786
SRGAN-MSE(official) 26.02 0.7397
SRGAN-MSE(mine) 28.52 0.7794

Results

Bicubic SRResNet-MSE(mine) Ground Truth
Alt text Alt text Alt text
Alt text Alt text Alt text
Alt text Alt text Alt text
Alt text Alt text Alt text
Alt text Alt text Alt text

Notes

  1. Before you train the networks , maybe you should change the directory path of training and testing data.
  2. Here offer a model named SRResNet-MSE_74s_sb_iter_180000.caffemodel which you can finetuning .
  3. Currently, the SRGAN-MSE doesn't work well , and it is still training and tuning.

Implementation Details

  1. According the paper "Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network" , I implementated the pixelshffuler_layer, which is added to reshape_layer.hpp && .cpp . If you want to use pixelshuffler_layer by yourself , just following this:
layer{
    name: "psx2_1"
    type: "Reshape"
    bottom: "conv_g35"
    top: "psx2_1"
    reshape_param {
       pixelshuffler: 2
    }
}
  1. For the GAN parts, the code borrows heavily from "在caffe 中实现Generative Adversarial Nets(二)" However , I add some new features to GAN parts. For examples , make gan_mode support the parameter "iter_size". When your gan_networks out of memory ,you can set the iter_sieze : 2 . For more details ,you can refference my srgan_is2_solver.prototxt and train_srgan_is2.prototxt
  2. For the utils of compute_psnr_ssim , the code borrows from this link https://github.com/ShenghaiRong/caffe-vdsr . But I modified the codes a lot.
  3. Because of the different caffe version, you may have problem when compiling the caffe. If so ,you can just modify the step function in your solver.cpp instead of directly using mine.

Plans

  • Finetuning the SRResNet-MSE model.
  • Properly train the SRGAN-MSE model.
  • Train the SRResNet-VGG networks.
  • Train the SRGAN-VGG networks.
  • Improve docs & instructions