No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Latest commit 6228e34 Feb 25, 2018
Permalink
Failed to load latest commit information.
code add code Dec 25, 2017
data add code Dec 25, 2017
examples add code Dec 25, 2017
models add code Dec 25, 2017
.gitignore add code Dec 25, 2017
LICENSE Initial commit Oct 20, 2017
README.md Update README.md Feb 26, 2018

README.md

StackGAN-v2

Pytorch implementation for reproducing StackGAN_v2 results in the paper StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks by Han Zhang*, Tao Xu*, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, Dimitris Metaxas.

Dependencies

python 2.7

Pytorch

In addition, please add the project folder to PYTHONPATH and pip install the following packages:

  • tensorboard
  • python-dateutil
  • easydict
  • pandas
  • torchfile

Data

  1. Download our preprocessed char-CNN-RNN text embeddings for birds and save them to data/
  • [Optional] Follow the instructions reedscot/icml2016 to download the pretrained char-CNN-RNN text encoders and extract text embeddings.
  1. Download the birds image data. Extract them to data/birds/
  2. Download ImageNet dataset and extract the images to data/imagenet/
  3. Download LSUN dataset and save the images to data/lsun

Training

  • Train a StackGAN-v2 model on the bird (CUB) dataset using our preprocessed embeddings:
    • python main.py --cfg cfg/birds_3stages.yml --gpu 0
  • Train a StackGAN-v2 model on the ImageNet dog subset:
    • python main.py --cfg cfg/dog_3stages_color.yml --gpu 0
  • Train a StackGAN-v2 model on the ImageNet cat subset:
    • python main.py --cfg cfg/cat_3stages_color.yml --gpu 0
  • Train a StackGAN-v2 model on the lsun bedroom subset:
    • python main.py --cfg cfg/bedroom_3stages_color.yml --gpu 0
  • Train a StackGAN-v2 model on the lsun church subset:
    • python main.py --cfg cfg/church_3stages_color.yml --gpu 0
  • *.yml files are example configuration files for training/evaluation our models.
  • If you want to try your own datasets, here are some good tips about how to train GAN. Also, we encourage to try different hyper-parameters and architectures, especially for more complex datasets.

Pretrained Model

Evaluating

  • Run python main.py --cfg cfg/eval_birds.yml --gpu 1 to generate samples from captions in birds validation set.
  • Change the eval_*.yml files to generate images from other pre-trained models.

Examples generated by StackGAN-v2

Tsne visualization of randomly generated birds, dogs, cats, churchs and bedrooms

Citing StackGAN++

If you find StackGAN useful in your research, please consider citing:

@article{Han17stackgan2,
  author    = {Han Zhang and Tao Xu and Hongsheng Li and Shaoting Zhang and Xiaogang Wang and Xiaolei Huang and Dimitris Metaxas},
  title     = {StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks},
  journal   = {arXiv: 1710.10916},
  year      = {2017},
}
@inproceedings{han2017stackgan,
Author = {Han Zhang and Tao Xu and Hongsheng Li and Shaoting Zhang and Xiaogang Wang and Xiaolei Huang and Dimitris Metaxas},
Title = {StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks},
Year = {2017},
booktitle = {{ICCV}},
}

Our follow-up work

References

  • Generative Adversarial Text-to-Image Synthesis Paper Code
  • Learning Deep Representations of Fine-grained Visual Descriptions Paper Code