Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis

Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang and Hongsheng Li.
Published in NeurIPS 2019.

Paper | Poster | Slides

Installation

Clone this repo.

git clone https://github.com/xh-liu/CC-FPSE.git
cd CC-FPSE/

This code requires PyTorch 1.1+ and python 3+. Please install dependencies by

pip install -r requirements.txt

The results reported in the paper is trained on 16 TITANX GPUs.

Dataset Preparation

Follow the dataset preparation process in SPADE.

Generating Images Using Pretrained Model

1. Download the pretrained models from Google Drive Folder, and extract it to 'checkpoints/'.

2. Generate images using the pretrained model with test_coco.sh, test_ade.sh, and test_cityscapes.sh.

3. The outputs images are stored at ./results/[type]_pretrained/ by default. You can view them using the autogenerated HTML file in the directory.

Training New Models

New models can be trained with train.sh. This is an example of training the model on one machine.

Citation

If you use this code for your research, please cite our papers.

@inproceedings{liu2019learning,
  title={Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis},
  author={Liu, Xihui and Yin, Guojun and Shao, Jing and Wang, Xiaogang and Li, Hongsheng},
  booktitle={Advances in Neural Information Processing Systems},
  year={2019}
}

Acknowledgments

This code borrows heavily from SPADE.