Skip to content
dev
Go to file
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 

README.md

A Survey on Rain Removal from Video and Single Image

Hong Wang, Yichen Wu, Minghan Li, Qian Zhao, and Deyu Meng

[Arxiv]

Citation

@article{WangA,
  title={A Survey on Rain Removal from Video and Single Image}, 
  author={Wang, Hong and Wu, Yichen and Li, Minghan and Zhao, Qian and Meng, Deyu}, 
  journal={arXiv preprint arXiv:1909.08326},
  year={2019}
}

Physical Properties of Raindrops

  • Gemometric Property
    • Terminal velocity of raindrops aloft (JAMC1969), Foote et al [PDF]
    • A new model for the equilibrium shape of raindrops (JAS1987), Beard et al. [PDF]
  • Brightness Property
    • Photometric model of a rain drop (Technical Report, Columbia University2004), Garg et al [PDF]
    • Vision and Rain (IJCV2007), Garg et al [Project][PDF]
  • Chromatic Property
    • Rain removal in video by combining temporal and chromatic properties (ICME2006), Zhang et al [Project][PDF]
  • Spatial and Temporal Propety
    • Simulation of rain in videos (TAS2003), Starik et al [PDF]
    • Pixel based temporal analysis using chromatic property for removing rain from videos (CIC2009), Liu et al [PDF]

Video Deraining Methods

  • Time Domain

    • Detection and removal of rain from videos (CVPR2004), Garg et al [Project][PDF]
    • When does camera see rain? (ICCV2005), Garg et al [Project][PDF]
    • Rain removal using kalman filter in video (ICSMA2008), Park et al [PDF]
    • Using the shape characteristics of rain to identify and remove rain from video (S+SSPR2008), Brewer et al [PDF]
    • The application of histogram on rain detection in video (JCIS2008), Zhao et al [PDF]
    • Rain or snow detection in image sequences through use of a histogram of orientation of streaks (IJCV2011), Bossu et al [PDF]
    • A probabilistic approach for detection and removal of rain from videos (IETE JR2011), Tripathi et al [PDF]
    • Video post processing: low latency spatiotemporal approach for detection and removal of rain (IET IP2012), Tripathi et al [PDF]
    • Removal of rain from videos: a review (SIVP2014), Tripathi et al [PDF]
    • Stereo video deraining and desnowing based on spatiotemporal frame warping (ICIP2014), Kim et al [PDF]
  • Frequency Domain

    • Spatio-temporal frequency analysis for removing rain and snow from videos (PACV2007), Barnum et al [Project] [PDF]
    • Analysis of rain and snow in frequency space (IJCV2010), Barnum et al [Project] [PDF]
  • Low Rank and Sparsity

    • A generalized low-rank appearance model for spatio-temporally correlated rain streaks (ICCV2013), Chen et al [PDF]
    • A rain pixel recovery algorithm for videos with highly dynamic scenes (TIP2013), Chen et al [PDF]
    • Video deraining and desnowing using temporal correlation and low-rank matrix completion (TIP2015), Kim et al [PDF] [Code]
    • Adherent raindrop modeling, detection and removal in video (TPAMI2016), You et al. [Project] [PDF]
    • Video desnowing and deraining based on matrix decomposition (CVPR2017), Ren et al [PDF] [Code]
    • A novel tensor-based video rain streaks removal approach via utilizing discriminatively intrinsic priors (CVPR2017), Jiang et al [PDF]
    • Should We encode rain streaks in video as deterministic or stochastic? (ICCV2017), Wei et al [PDF] [Code]
    • A directional global sparse model for single image rain removal (AMM2018), Deng et al [PDF] [Code]
    • Video rain streak removal by multiscale convolutional sparse coding (CVPR2018), Li et al [Project] [PDF] [Code]
    • Fastderain: A novel video rain streak removal method using directional gradient priors (TIP2019), Jiang et al [PDF] [Code]
  • Deep Learning

    • Robust video content alignment and compensation for rain removal in a cnn framework (CVPR2018), Chen et al [PDF] [Code]
    • Erase or fill? deep joint recurrent rain removal and reconstruction in videos (CVPR2018), Liu et al. [Project][PDF] [Code]
    • D3R-Net: dynamic routing residue recurrent network for video rain removal (TIP2018), Liu et al. [PDF]
    • Frame consistent recurrent video deraining with dual-level flow (CVPR2019), Yang et al. [Code]
    • Self-Learning Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence(CVPR2020), Yang et al.[PDF][Supplementray Materials] [Code]
  • Reivew paper

    • Removal of rain from videos: a review (SIVP2014), Tripathi et al [PDF]
    • A Survey on Rain Removal from Video and Single Image (Arxiv2019), Wang et al. [PDF] [Code]

Single Image Deraining Methods

  • Filter based methods

    • Guided image filtering (ECCV2010), He et al. [Project] [PDF] [Code]
    • Removing rain and snow in a single image using guided filter (CSAE2012), Xu et al. [PDF]
    • An improved guidance image based method to remove rain and snow in a single image (CIS2012), Xu et al. [PDF]
    • Single-image deraining using an adaptive nonlocal means filter (ICIP2013), Kim et al. [PDF]
    • Single-image-based rain and snow removal using multi-guided filter (NIPS2013), Zheng et al. [PDF]
    • Single image rain and snow removal via guided L0 smoothing filter (Multimedia Tools and Application2016), Ding et al. [PDF]
  • Prior based methods

    • Automatic single-image-based rain streaks removal via image decomposition (TIP2012), Kang et al [PDF] [Code]
    • Self-learning-based rain streak removal for image/video (ISCS2012), Kang et al. [PDF]
    • Single-frame-based rain removal via image decomposition (ICA2013), Fu et al. [PDF]
    • Exploiting image structural similarity for single image rain removal (ICIP2014), Sun et al. [PDF]
    • Visual depth guided color image rain streaks removal using sparse coding (TCSVT2014), Chen et al [PDF]
    • Removing rain from a single image via discriminative sparse coding (ICCV2015), Luo et al [PDF] [Code] pwd: d229
    • Rain streak removal using layer priors (CVPR2016), Li et al [PDF] [Code]
    • Single image rain streak decomposition using layer priors (TIP2017), Li et al [PDF]
    • Error-optimized dparse representation for single image rain removal (IEEE TIE2017), Chen et al [PDF]
    • A hierarchical approach for rain or snow removing in a single color image (TIP2017), Wang et al. [PDF]
    • Joint bi-layer optimization for single-image rain streak removal (ICCV2017), Zhu et al. [PDF]
    • Convolutional sparse and low-rank codingbased rain streak removal (WCACV2017), Zhang et al [PDF]
    • Joint convolutional analysis and synthesis sparse representation for single image layer separation (CVPR2017), Gu et al [PDF] [Code]
    • Single image deraining via decorrelating the rain streaks and background scene in gradient domain (PR2018), Du et al [PDF]
  • Deep Learning

    • Restoring an image taken through a window covered with dirt or rain (ICCV2013), Eigen et al. [Project] [PDF] [Code]
    • Attentive generative adversarial network for raindrop removal from a single image (CVPR2018), Qian et al [Project] [PDF]
    • Clearing the skies: A deep network architecture for single-image rain streaks removal (TIP2017), Fu et al. [Project] [PDF] [Code]
    • Removing rain from single images via a deep detail network (CVPR2017), Fu et al. [Project] [PDF] [Code]
    • Image de-raining using a conditional generative adversarial network (Arxiv2017), Zhang et al [PDF] [Code]
    • Deep joint rain detection and removal from a single image (CVPR2017), Yang et al.[Project] [PDF] [Code]
    • Residual guide feature fusion network for single image deraining (ACMMM2018), Fan et al. [Project] [PDF]
    • Fast single image rain removal via a deep decomposition-composition network (Arxiv2018), Li et al [Project]) [PDF] [Code]
    • Density-aware single image de-raining using a multi-stream dense network (CVPR2018), Zhang et al [PDF] [Code]
    • Recurrent squeeze-and-excitation context aggregation net for single image deraining (ECCV2018), Li et al. [PDF] [Code]
    • Rain streak removal for single image via kernel guided cnn (Arxiv2018), Wang et al [PDF]
    • Physics-based generative adversarial models for image restoration and beyond (Arxiv2018), Pan et al [PDF]
    • Learning dual convolutional neural networks for low-level vision (CVPR2018), Pan et al [Project] [PDF] [Code]
    • Non-locally enhanced encoder-decoder network for single image de-raining (ACMMM2018), Li et al [PDF] [Code]
    • Single image rain removal via a deep decomposition-composition network (CVIU2019), Li et al.
    • Unsupervised single image deraining with self-supervised constraints (ICIP2019), Jin et al [PDF]
    • Residual multiscale based single image deraining (BMVC2019), Zheng et al.
    • Erl-net: Entangled representation learning for single image de-raining (ICCV2019), Wang et al. [code]
    • Uncertainty guided multi-scale residual learning-using a cycle spinning cnn for single image de-raining (CVPR2019), Rajeev Yasarla et al.[Code]
    • Heavy rain image restoration: Integrating physics model and conditional adversarial learning (CVPR2019), Li et al.[Code]
    • Progressive image deraining networks: A better and simpler baseline (CVPR2019), Ren et al [PDF] [Code]
    • Spatial attentive single-image deraining with a high quality real rain dataset (CVPR2019), Wang et al [Project] [PDF] [Code]
    • Lightweight pyramid networks for image deraining (TNNLS2019), Fu et al [PDF] [Code]
    • Joint rain detection and removal from a single image with contextualized deep networks (TPAMI2019), Yang et al [PDF] [Code]
    • Scale-free single image deraining via visibility-enhanced recurrent wavelet learning (TIP2019), Yang et al.[PDF]
    • Towards scale-free rain streak removal via selfsupervised fractal band learning (AAAI2020), Yang et al.[Code]
    • Structural Residual Learning for Single Image Rain Removal(Arxiv2020), Wang et al. [PDF]
    • All in One Bad Weather Removal Using Architectural Search (CVPR2020), Li et al.[PDF]
    • Syn2Real Transfer Learning for Image Deraining Using Gaussian Processes(CVPR2020), Rajeev Yasarla et al. [Code]
    • Multi-Scale Progressive Fusion Network for Single Image Deraining(CVPR2020), Jiang et al. [Code]
    • Detail-recovery Image Deraining via Context Aggregation Networks(CVPR2020), Deng et al.[Code]
    • Variational image deraining(WACV2020), Du et al.[PDF]
  • Joint Model-driven and Data-driven

    • Deep Layer Prior Optimization for Single Image Rain Streaks Removal (ICASSP2018), Liu et al [PDF]
    • Learning bilevel layer priors for single image rain streaks removal (SP Letters 2018), Mu et al [PDF]
    • Semi-supervised transfer learning for image rain removal (CVPR2019), Wei et al [PDF] [Code]
    • Knowledge-driven deep unrolling for robust image layer separation (TNNLS2019), Liu et al.
    • A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020), Wang et al [PDF][Supplementary Materials [Code]
  • Reivew paper

    • Single image deraining: A comprehensive benchmark analysis(CVPR2019), Li et al.[PDF] [Code]
    • A Survey on Rain Removal from Video and Single Image (Arxiv2019), Wang et al. [PDF] [Code]
    • Single image deraining: From model-based to data-driven and beyond(TPAMI2020), Yang et al.[Code]

Datasets and Discriptions

*We note that:

i. RainTrainL/Rain100L and RainTrainH/Rain100H are synthesized by Yang Wenhan. Rain12600/Rain1400 is from Fu Xueyang and Rain12 is from Li Yu.

ii. In video experiment, the rain-removed results of the deep learning method are provided by the author Yang Wenhan. Really thanks!

iii. In single image experiment, we seperately retrain all the recent state-of-the-art methods via the three training datasets: RainTrainL(200 input/clean image pairs), RainTrainH(1800 pairs), and Rain12600(12600 pairs), and then evaluate their rain removal performance based on the correponding test datasets: Rain100L(100 pairs), Rain100H(100 pairs), and Rain1400(1400 pairs). Besides, the trained model obtained by RainTrainL is adpoted to predict rain-removed results of Rain12(12 pairs). Moreover, we utilize the Internet-Data(147 input images) and SPA-Data(1000 pairs) to compare the generalization ability.

iiii. In single image experiment, when training the semi-supervised method--SIRR, we always utilize Internet-Data as unsupervised samples.

Image Quality Metrics

*Please note that all quantitative results in our survey paper are computed based on Y channel.

Contact

If you have any question, please feel free to concat Hong Wang (Email: hongwang01@stu.xjtu.edu.cn).

About

categorize current rain removal methods and fairly evaluate their deraining ability based on diverse benchmark datasets

Resources

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.