Skip to content
Official PyTorch Implementation of "Example-Guided Style-Consistent Image Synthesis from Semantic Labeling" - CVPR 2019
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
img
models
new_scripts
options
util
.gitignore
LICENSE.txt
README.md
_config.yml init Jun 18, 2019
cp_data.py
encode_features.py
expand_val_test.py
generate_data_face_forensics.py
judge_face.py
precompute_feature_maps.py
process.py
run_engine.py
temp_test.py
test.py
test_all.py
test_con.py
test_con_bak.py
test_delta.txt
test_face.py init Jun 18, 2019
test_mface.py
test_pose.py
test_seg.py
train.py
train_con.py
train_face.py
train_mface.py
train_pose.py
train_seg.py
val_gen.py
vis_bdd.py
vis_delta.txt
vis_face.py
vis_pose.py

README.md

Example-Guided Style-Consistent Image Synthesis from Semantic Labeling

Paper

Example-Guided Style-Consistent Image Synthesis from Semantic Labeling
Miao Wang1, Guo-Ye Yang2, Ruilong Li2, Run-Ze Liang2, Song-Hai Zhang2, Peter M. Hall3 and Shi-Min Hu2,1
1State Key Laboratory of Virtual Reality Technology and Systems, Beihang University
2Department of Computer Science and Technology, Tsinghua University, Beijing
3University of Bath
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019

Prerequisites

  • Linux
  • Python 3
  • NVIDIA GPU (12G or 24G memory) + CUDA cuDNN
  • pytorch==0.4.1
  • numpy
  • ...

Tasks

Sketch2Face

Task name: face

We use the real videos in the FaceForensics dataset, which contains 854 videos of reporters broadcasting news. We localize facial landmarks, crop facial regions and resize them to size 256×256. The detected facial landmarks are connected to create face sketches.

Pose2Dance

Task name: pose

We download 150 solo dance videos from YouTube, crop out the central body regions and resize them to 256×256. We evenly split each video into the first part and the second part along the time-line, then sample training data only from the first parts and sample testing data only from the second parts of all the videos. The the labels are created using concatenated pre-trained DensePose and OpenPose pose detection results.

SceneParsing2StreetView

Task name: scene

We use the BDD100k dataset to synthesize street view images from pixelwise semantic labels (i.e. scene parsing maps). We use the state-of-the-art scene parsing network DANet to create labels.

Getting Started

Installation

git clone [this project]
mkdir FaceForensics
download FaceForensics dataset to FaceForensics/datas
cd Example-Guided-Image-Synthesis
python process.py
mkdir datasets
python generate_data_face_forensics.py --source_path '../FaceForensics/out_data' --target_path './datasets/FaceForensics3/' --same_style_rate 0.3 --neighbor_size 10 --A_repeat_num 50 --copy_data
download data.zip at ../
unzip data.zip
mv ../data/checkpoints Example-Guided-Image-Synthesis/

Training

new_scripts/train_[Task name].sh

Testing

new_scripts/test_[Task name].sh

Results

Face

Dance

Scene

Citation

If you find this useful for your research, please cite the following paper.

@InProceedings{pix2pixSC2019,
author = {Wang, Miao and Yang, Guo-Ye and Li, Ruilong and Liang, Run-Ze and Zhang, Song-Hai and Hall, Peter. M and Hu, Shi-Min},
title = {Example-Guided Style-Consistent Image Synthesis from Semantic Labeling},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}
You can’t perform that action at this time.