TherISuRNet in Tensorflow

This is an official implementation of the our CVPRW 2020 paper "TherISuRNet - A Computationally Efficient Thermal Image Super-Resolution Network" using the Tensorflow (version 1.12.0). Paper can be downloaded from here

Authors: Vishal Chudasama, Heena Patel, Kalpesh Prajapati, Kishor Upla, Raghavendra Ramachandra, Kiran Raja, Christoph Busch

Network design:

Residual network (ResBlock):

Usage

1. Download the PBVS Training dataset from here

• Train Data bicubic downscaling x4 (LR images)

• Train Data (HR images)

• Validation Data bicubic downscaling x4 (LR images)

• Validation Data (HR images)

  Put the downloaded dataset in the train_lr, train_hr, val_lr and val_hr folders like this

...
│
├── train_lr
│     ├── 1.png
│     ├── 2.png
│     ├── ...
├── train_hr
│     ├── 1.png
│     ├── 2.png
│     ├── ...
├── val_lr
│     ├── 1.png
│     ├── 2.png
│     ├── ...
├── val_hr
│     ├── 1.png
│     ├── 2.png
│     ├── ...
├── main.py
├── model.py
...

2. Download the imagenet-vgg-verydeep-19.mat file. (Note - Only if you want to use Contextual loss [3] in training process)

3. Train TherISuRNet model.

python main.py

4. After training, inference can be performed.

Download the testing LR and HR images from here (Note: Bicubic Interpolated Images are also attached herewith)

PBVS Validation dataset : 50 number of images [1]

FLIR Validation dataset : 1366 number of images

KAIST Validation dataset : 500 preprocessed images

Put above test images and/or your own test images that you want to evaluate into a test folder

example:

...
│
├── main.py
├── pred.py
├── model
│     ├── checkpoint
│     ├── model.ckpt-100000
│     ├── model.ckpt-100000
│     └── model.ckpt-100000
├── test
│     ├── yourpic1.png
│     ├── favpic.jpg
│     ...
│     └── smallpic.png
...

and run the following command.

python pred.py test

5. To calculate the corresponding PSNR and SSIM measures

python PSNR_SSIM.py

Result examples

The SR results of the TherISuRNet model can be downloaded from here.

Computationally Efficiency Comparison:

Quantitative Comparison:

References

1. Rafael E. Rivadeneira, Angel D. Sappa, and Boris X. Vintimilla. Thermal image super-resolution: a novel architecture and dataset. In International Conference on Computer Vision Theory and Applications, pages 1–2, 2020.
2. Purbaditya Bhattacharya, Jorg Riechen, and Udo Zolzer. Infrared image enhancement in maritime environment with convolutional neural networks. In VISIGRAPP, 2018.
3. Mechrez, Roey and Talmi, Itamar and Zelnik-Manor, Lihi, The Contextual Loss for Image Transformation with Non-Aligned Data, arXiv preprint arXiv:1803.02077, 2018.

If you use this repositary, then please cite our paper in your publications as

V. Chudasama et al., "TherISuRNet - A Computationally Efficient Thermal Image Super-Resolution Network," 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 2020, pp. 388-397, doi: 10.1109/CVPRW50498.2020.00051.

Bibtex file:

@INPROCEEDINGS{TherISuRNet,
author={V. {Chudasama} and H. {Patel} and K. {Prajapati} and K. {Upla} and R. {Ramachandra} and K. {Raja} and C. {Busch}},
booktitle={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
title={TherISuRNet - A Computationally Efficient Thermal Image Super-Resolution Network},
year={2020},
pages={388-397},
doi={10.1109/CVPRW50498.2020.00051}}

We are thankful to itsuki8914 for their ESRGAN code in tensorflow on which we made this implementation. (https://github.com/itsuki8914/ESRGAN-TensorFlow)