Skip to content

hhsinping/svs

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

[ECCV 2020] Semantic View Synthesis

[Paper] [Project Website] [Google Colab]

We tackle a new problem of semantic view synthesis --- generating free-viewpoint rendering of a synthesized scene using a semantic label map as input. We build upon recent advances in semantic image synthesis and view synthesis for handling photographic image content generation and view extrapolation. Direct application of existing image/view synthesis methods, however, results in severe ghosting/blurry artifacts. To address the drawbacks, we propose a two-step approach. First, we focus on synthesizing the color and depth of the visible surface of the 3D scene. We then use the synthesized color and depth to impose explicit constraints on the multiple-plane image (MPI) representation prediction process. Our method produces sharp contents at the original view and geometrically consistent renderings across novel viewpoints. The experiments on numerous indoor and outdoor images show favorable results against several strong baselines and validate the effectiveness of our approach.

Semantic View Synthesis
Hsin-Ping Huang, Hung-Yu Tseng, Hsin-Ying Lee, and Jia-Bin Huang
In European Conference on Computer Vision (ECCV), 2020.

Quick start

conda create -n svs python=3.7
source activate svs
conda install pytorch==1.1.0 torchvision==0.3.0
pip install scikit-image==0.15.0 dill==0.2.9 moviepy==1.0.1

git clone https://github.com/hhsinping/svs.git
cd svs
bash download_model.sh
# put your own semantic maps into inputs folder 
# OR bash download_input.sh and select maps from labels folder
bash test.sh

Paper

@inproceedings{SVS,
  author       = "Huang, Hsin-Ping and Tseng, Hung-Yu and Lee, Hsin-Ying and Huang, Jia-Bin",
  title        = "Semantic View Synthesis",
  booktitle    = "European Conference on Computer Vision (ECCV)",
  year         = "2020"
}

Acknowledgments

Our work builds upon

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published