Single Image Deraining via Recurrent Hierarchy Enhancement Network (ACM'MM19)

[pdf][code][DerainZoo]

Youzhao Yang, and Hong Lu; School of Computer Science, Fudan University

Abstract

Single image deraining is an important problem in many computer vision tasks since rain streaks can severely hamper and degrade the visibility of images. In this paper, we propose a novel network named Recurrent Hierarchy Enhancement Network (ReHEN) to remove rain streaks from rainy images stage by stage. Unlike previous deep convolutional network methods, we adopt a Hierarchy Enhancement Unit (HEU) to fully extract local hierarchical features and generate effective features. Then a Recurrent Enhancement Unit (REU) is added to keep the useful information from HEU and benefit the rain removal in the later stages. To focus on different scales, shapes, and densities of rain streaks adaptively, Squeeze-and-Excitation (SE) block is applied in both HEU and REU to assign different scale factors to high-level features. Experiments on five synthetic datasets and a real-world rainy image set show that the proposed method outperforms the state- of-the-art methods considerably. The source code is available at https://github.com/nnUyi/ReHEN .

Dataset

Synthetic Datasets

Datasets	#train	#test	label
Rain100L	200	100	rain mask & rain map
Rain100H	1800	100	rain mask & rain map
Rain800	700	100	-
Rain1200	12000	1200	rain density
Rain1400	12600	1400	-

Real-World Datasets

Datasets	#train	#test	label
Real-World	-	67	-

Pre-trained Model

We note that these models is trained on NVIDIA GeForce GTX1070:

Datasets	Pre-trained model
Rain100L	Rain100L model
Rain100H	Rain100H model
Rain800	Rain800 model
Rain1200	Rain1200 model TAB
Rain1400	Rain1400 model TAB

Requirements

• python 3.6.5
• opencv 3.4.2
• numpy 1.14.3
• argparse 1.1
• tensorflow_gpu >=1.4.0 & < 2.0

Usages

• Clone this repo

   $ git clone https://github.com/nnUyi/ReHEN
   $ cd ReHEN
   $ cd codes

• Train

   $ CUDA_VISIBLE_DEVICES=0 python main.py --is_training True
                                           --train_dataset Rain100L
                                           --test_dataset Rain100L
                                           --trainset_size 200
                                           --testset_size 100
                                           --iterations 60000
                                           --lr 0.005
                                           --batch_size 32

• Test

   $ CUDA_VISIBLE_DEVICES=0 python main.py --is_testing True
                                           --train_dataset Rain100L
                                           --test_dataset Rain100L
                                           --trainset_size 200
                                           --testset_size 100
                                           --batch_size 32

• Where should your datasets place?
  • In the 34rd and 35rd lines of settings.py, you can see the --data_path and --test_path settings. You should download the datasets (or putting your own dataset) in the target path. And you should add your dataset format in train_dic & test_dic in the 50th and 59th lines of settings.py.

Results

Visualization of Hidden Statuses

Average PSNR and SSIM values on five synthetic datasets

Methods	--	--	Rain100L	Rain100H	Rain800	Rain1200	Rain1400
--	Params	Running Time (s)	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM	PSNR/SSIM
JORDER (CVPR'17)[1]	369,792	0.268	36.11/0.970	22.15/0.674	22.24/0.776	24.32/0.862	27.55/0.853
DID-MDN (CVPR'18)[2]	372,839	0.532	25.70/0.858	17.39/0.612	21.89/0.795	27.95/0.908	27.99/0.869
DualCNN (CVPR'18)[3]	687,008	20.19	26.87/0.860	14.23/0.468	24.11/0.821	23.38/0.787	24.98/0.838
RESCAN (ECCV'18)[4]	134,424	0.750	36.64/0.975	26.45/0.846	24.09/0.841	29.95/0.884	28.57/0.891
OURS (ACMMM'19)	298,263	0.531	37.41/0.980	27.97/0.864	26.96/0.854	32.64/0.914	31.33/0.918

References

[1] Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan, “Deep joint rain detection and removal from a single image,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1357–1366.

[2] He Zhang and Vishal M. Patel, “Density-aware single image deraining using a multi-stream dense network,” in The IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 695–704.

[3] Jinshan Pan, Sifei Liu, Deqing Sun, Jiawei Zhang, Yang Liu, Jimmy Ren, Zechao Li, Jinhui Tang, Huchuan Lu, and Yu-Wing Tai, “Learning dual convolutional neural networks for low-level vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3070–3079.

[4] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha, “Recurrent squeeze-and-excitation context aggregation net for single image deraining,” in The European Conference on Computer Vision (ECCV), 2018, pp. 262–277.

Acknowledgements

• This work was supported in part by National Natural Science Foundation of China (No. U1509206).

Citation

If you find the resource useful, please cite the following:

   @inproceedings{yang2019single,
      title={Single Image Deraining via Recurrent Hierarchy Enhancement Network},
      author={Yang, Youzhao and Lu, Hong},   
      booktitle={ACM International Conference on Multimedia (ACM'MM)},
      year={2019}
   }