Skip to content

A TensorFlow-based image super-resolution model considering both quantitative and perceptual quality

License

Notifications You must be signed in to change notification settings

idearibosome/tf-perceptual-eusr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

4PP-EUSR

Four-pass perceptual super-resolution with enhanced upscaling

SRZoo

Introduction

This repository contains a TensorFlow-based implementation of 4PP-EUSR ("Deep learning-based image super-resolution considering quantitative and perceptual quality"), which considers both the quantitative (e.g., PSNR) and perceptual quality (e.g., NIQE) of the upscaled images. Our method won the 2nd place and got the highest human opinion score for Region 2 in the 2018 PIRM Challenge on Perceptual Image Super-resolution at ECCV 2018.

BSD100 - 37073 ※ The perceptual index is calculated by "0.5 * ((10 - Ma) + NIQE)", which is used in the PIRM Challenge. Lower is better.

Followings are the performance comparison evaluated on the BSD100 dataset.

BSD100 PSNR vs. NIQE

Method PSNR (dB) (↓) Perceptual Index
EDSR 27.796 5.326
MDSR 27.771 5.424
EUSR 27.674 5.307
SRResNet-MSE 27.601 5.217
4PP-EUSR (PIRM Challenge) 26.569 2.683
SRResNet-VGG22 26.322 5.183
SRGAN-MSE 25.981 2.802
Bicubic interpolation 25.957 6.995
SRGAN-VGG22 25.697 2.631
SRGAN-VGG54 25.176 2.351
CX 24.581 2.250

Please cite following papers when you use the code, pre-trained models, or results:

  • J.-H. Choi, J.-H. Kim, M. Cheon, J.-S. Lee: Deep learning-based image super-resolution considering quantitative and perceptual quality. Neurocomputing (In Press) [Paper] [arXiv]
@article{choi2018deep,
  title={Deep learning-based image super-resolution considering quantitative and perceptual quality},
  author={Choi, Jun-Ho and Kim, Jun-Hyuk and Cheon, Manri and Lee, Jong-Seok},
  journal={Neurocomputing},
  year={2019},
  publisher={Elsevier}
}
  • J.-H. Kim, J.-S. Lee: Deep residual network with enhanced upscaling module for super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 913-921 (2018) [Paper]
@inproceedings{kim2018deep,
  title={Deep residual network with enhanced upscaling module for super-resolution},
  author={Kim, Jun-Hyuk and Lee, Jong-Seok},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
  year={2018}
}

Dependencies

  • Python 3.6+
  • TensorFlow 1.11+ (<2.0)

Testing pre-trained models

※ PIRM Challenge version is available on SRZoo.

Generating upscaled images from the trained models can be done by test/test.py. Following are the brief instructions.

  1. Download and copy the trained model available in Downloads section to the test/ folder.
  2. Place the low-resolution images (PNG only) to the test/LR/ folder.
  3. Run python test.py --model_name [model file name]. For example, if you downloaded the PIRM Challenge version of our pre-trained model, run python test.py --model_name 4pp_eusr_pirm.pb.
  4. The upscaled images will be available on the test/SR/ folder.

Please run python test.py --help for more information.

Training EUSR, score predictors, and 4PP-EUSR

Please refer to README.md on the train/ folder for more information.

Downloads

Pre-trained models:

Results (Set5, Set14, BSD100, PIRM):

About

A TensorFlow-based image super-resolution model considering both quantitative and perceptual quality

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published