Text To Image Synthesis

virtualenv --system-site-packages -p python3 tensorflow
source tensorflow/bin/activate
pip3 install --upgrade tensorflow-gpu
pip3 install tensorlayer
pip3 install nltk
pip3 install imageio

This is an experimental tensorflow implementation of synthesizing images. The images are synthesized using the GAN-CLS Algorithm from the paper Generative Adversarial Text-to-Image Synthesis. This implementation is built on top of the excellent DCGAN in Tensorflow.

Image Source : Generative Adversarial Text-to-Image Synthesis Paper

Requirements

• TensorFlow 1.0+
• TensorLayer 1.4+
• NLTK : for tokenizer

Datasets

• The model is currently trained on the flowers dataset. Download the images from here and save them in 102flowers/102flowers/*.jpg. Also download the captions from this link. Extract the archive, copy the text_c10 folder and paste it in 102flowers/text_c10/class_*.

N.B You can downloads all data files needed manually or simply run the downloads.py and put the correct files to the right directories.

python downloads.py

Codes

• downloads.py download Oxford-102 flower dataset and caption files(run this first).
• data_loader.py load data for further processing.
• train_txt2im.py train a text to image model.
• utils.py helper functions.
• model.py models.

References

• Generative Adversarial Text-to-Image Synthesis Paper
• Generative Adversarial Text-to-Image Synthesis Torch Code
• Skip Thought Vectors Paper
• Skip Thought Vectors Code
• Generative Adversarial Text-to-Image Synthesis with Skip Thought Vectors TensorFlow code
• DCGAN in Tensorflow

Results

• the flower shown has yellow anther red pistil and bright red petals.
• this flower has petals that are yellow, white and purple and has dark lines
• the petals on this flower are white with a yellow center
• this flower has a lot of small round pink petals.
• this flower is orange in color, and has petals that are ruffled and rounded.
• the flower has yellow petals and the center of it is brown
• this flower has petals that are blue and white.
• these white flowers have petals that start off white in color and end in a white towards the tips.

License

Apache 2.0