Joint VAE Model API
Variational autoencoder for images and text
Describes the code used to create results for the following paper:
Vedantam, Ramakrishna, Ian Fischer, Jonathan Huang, and Kevin Murphy. 2017. Generative Models of Visually Grounded Imagination. arXiv [cs.LG]. arXiv. http://arxiv.org/abs/1705.10762.
NOTE: All scripts should be run from the root directory of the project.
Install basic dependencies.
cd to the root directory and run the following command.
This sets up the python virtual environment for the project, and downloads necessary files for MNIST-A experiments.
Additional Data Setup
To create your own MNISTA dataset see
Set appropriate paths in
To download and process CELEBA dataset run
And set appropriate paths in
See scripts/ for example uses of different models/ experiments reported in the paper.
iclr_comp_mnista_fresh.sh: Experiments on compositional split of MNISTA. 2)
iclr_mnista_fresh.sh: Experiments on iid split of MNISTA. 3)
celeba_training.sh: Experiments on CelebA dataset.
Running above scripts uses slurm to run/launch jobs on clusters, so only run the above if you have access to a cluster with slurm installed on it. If you dont have access to slurm simply run the corresponding python commands in the above scripts with the appropriate parameters.
Each experiment is associated with three jobs:
imeval (imagination evaluation).
Example commands to run experiments with slurm:
source scripts/iclr_mnista_fresh.sh ''(to run all 3 jobs for every experiment)
source scripts/iclr_mnista_fresh.sh train(to launch only training jobs)
source scripts/iclr_mnista_fresh.sh eval(to launch only eval jobs)
source scripts/iclr_mnista_fresh.sh imeval(to launch only imagination eval jobs)
If you dont have access to slurm, you can see what command line arguments are used for running experiments (in the above scripts) and run those commands in bash.
See the ipython notebook
experiments/iclr_results_aggregate.ipynb on how to view
the imagination results after running imeval (imagination evaluation) jobs, post
- Ramakrishna Vedantam
- Hernan Moraldo
- Ian Fischer
This project is not an official Google project. It is not supported by Google and Google specifically disclaims all warranties as to its quality, merchantability, or fitness for a particular purpose.
See how to contribute.