Skip to content
/ Mono3D Public

Source code for "Mononizing Binocular Videos" (SIGGRAPH Asia 2020)

License

Notifications You must be signed in to change notification settings

wbhu/Mono3D

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

18 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Mononizing Binocular Videos

ACM Transactions on Graphics (SIGGRAPH Asia 2020 issue), Vol. 39, No. 6, December 2020, pp. 228:1--228:16.

[ Project Webpage ] [ arXiv ] [ Video ]

Mono3D is the implementation of mono-nizing binocular videos into a regular monocular video with the stereo information implicitly encoded, such that the original binocular videos can be restored with high quality.

teaser

Online demo

[ Mononized view ] [ Restored left view ] [ Restored right view ]

Environment

Please refer to env.yaml.

  • Please carefully install following two packages with specific version, because the pretrained model is based on that version and newer versions are incompatible.
- mmcv==0.6.2
- mmdet==2.2.1 (build from source)

Dataset

We cannot release the whole 3D movie dataset due to copyright issues. But the binocular image dataset and part of the binocular video dataset used in the paper are publicly available: [ Flickr1024 ] and [ Inria ].

Prepare Flickr1024 for training the image version model

  1. Download Flickr1024 from the website: https://yingqianwang.github.io/Flickr1024/
  2. Download data list from https://drive.google.com/drive/folders/14oeXizbqTCxbmkZblt7YbWjaU2IIqNJf?usp=sharing
  3. Organise the dataset as following (${DATASET is the root dir for maintaining our dataset}):
${DATASET}  
|-- Flickr1024  
|   |-- Train  
|   |-- |-- 0001_L.png  
|   |   |-- 0001_R.png
|   |   |-- 0002_L.png  
|   |   |-- 0002_R.png
|   |   |-- ...
|   |-- Validation  
|   |-- |-- 0001_L.png  
|   |   |-- 0001_R.png
|   |   |-- 0002_L.png  
|   |   |-- 0002_R.png
|   |   |-- ...
|   |-- Test 
|   |-- |-- 0001_L.png  
|   |   |-- 0001_R.png
|   |   |-- 0002_L.png  
|   |   |-- 0002_R.png
|   |   |-- ...
|   |-- list  
|   |-- |-- train.txt
|   |   |-- val.txt
|   |   |-- test.txt  

Demo

$ PYTHONPATH=. python main/demo.py --left ./imgs/demo_L.png

Training

$ sh scripts/train.sh mono3d_img config/Flickr1024/mono3d_img.yaml

Evaluation

Evaluation on the testing set of Flickr1024

$ sh scripts/test.sh mono3d_img config/Flickr1024/mono3d_img.yaml

Copyright and License

You are granted with the LICENSE for both academic and commercial usages.

Acknowledgments

Thanks to Yingqian Wang for releasing the great dataset, Flickr1024.

Citation

@article{hu-2020-mononizing,
        author   = {Wenbo Hu and Menghan Xia and Chi-Wing Fu and Tien-Tsin Wong},
        title    = {Mononizing Binocular Videos},
        journal  = {ACM Transactions on Graphics (SIGGRAPH Asia 2020 issue)},
        month    = {December},
        year     = {2020},
        volume   = {39},
        number   = {6},
        pages    = {228:1-228:16}
    }

Releases

No releases published

Packages

No packages published