Skip to content

occlusion inpainting for unsupervised optical flow estimation

Notifications You must be signed in to change notification settings

coolbeam/OIFlow

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

[TIP2021] OIFlow: Occlusion-Inpainting Optical Flow Estimation by Unsupervised Learning

Shuaicheng Liu1,2, Kunming Luo2, Nianjin Ye2, Chuan Wang2, Jue Wang2, Bing Zeng1

1. University of Electronic Science and Technology of China

2. Megvii Research

This is the official implementation of paper OIFlow: Occlusion-Inpainting Optical Flow Estimation by Unsupervised Learning, IEEE Transactions on Image Processing, 2021

inpainting_visualization

Abstract

Occlusion is an inevitable and critical problem in unsupervised optical flow learning. Existing methods either treat occlusions equally as non-occluded regions or simply remove them to avoid incorrectness. However, the occlusion regions can provide effective information for optical flow learning. In this paper, we present OIFlow, an occlusion-inpainting framework to make full use of occlusion regions. Specifically, a new appearance-flow network is proposed to inpaint occluded flows based on the image content. Moreover, a boundary dilated warp is proposed to deal with occlusions caused by displacement beyond the image border. We conduct experiments on multiple leading flow benchmark datasets such as Flying Chairs, KITTI and MPISintel, which demonstrate that the performance is significantly improved by our proposed occlusion handling framework.

Overview

overview (a) reference image. (b) ground truth flow. (c)(d) results without / with the optical flow refinement. (e) detected occlusion map. (f) a zoom-in window with mask (dark region) overlaid on the image. Here an appearance flow is learned on the image domain, which is further used to inpaint the occluded regions by the non-occluded ones. (g) the appearance flow of (f) is applied to guide the propagation in the flow field. (h) The propagated results in the zoom-in region.

Citation

If you think this work is helpful, please cite

    @article{liu2021oiflow,
      title={OIFlow: occlusion-inpainting optical flow estimation by unsupervised learning},
      author={Liu, Shuaicheng and Luo, Kunming and Ye, Nianjin and Wang, Chuan and Wang, Jue and Zeng, Bing},
      journal={IEEE Transactions on Image Processing},
      volume={30},
      pages={6420--6433},
      year={2021},
      publisher={IEEE}
    }

Acknowledgement

Part of our codes are adapted from IRR-PWC and UnFlow, we thank the authors for their contributions.

About

occlusion inpainting for unsupervised optical flow estimation

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published