Skip to content

sisl/EvidentialSparsification

main
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Evidential Sparsification of Multimodal Latent Spaces in CVAEs

This repository contains the code for the paper: "Evidential Sparsification of Multimodal Latent Spaces in Conditional Variational Autoencoders" by Masha Itkina, Boris Ivanovic, Ransalu Senanayake, Mykel J. Kochenderfer, and Marco Pavone, presented at NeurIPS 2020.

Our method uses evidential theory to sparsify the discrete latent space of a CVAE, while maintaining the distributional multimodality. The code runs the qualitative and quantitative training iteration experiments for the tasks of image generation and behavior prediction. The required dependencies are listed in dependencies.txt.

To run the MNIST, FashionMNIST, and NotMNIST image generation experiments, please run the following files:

run_mnist.sh run_fashion.sh run_notmnist.sh

Note that the NotMNIST data has to be downloaded from: http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html and placed into "NotMNIST/data/archive/notMNIST_large/notMNIST_large/".

For the VQVAE image generation experiments, the miniImageNet dataset should be downloaded from: https://drive.google.com/file/d/0B3Irx3uQNoBMQ1FlNXJsZUdYWEE/view. The "train.csv", "val.csv", and "test.csv" files should be placed into VQVAE/data/miniimagenet, and the image data should be copied to VQVAE/data/miniimagenet/images. As an example, Table 1 in the paper can be reproduced using the following commands:

For training:

run_vqvae.sh

If you want to use the pre-trained models, download them from: https://drive.google.com/drive/folders/1MOM9WZQd3tB-aG-hlwhHZFrDehJqLsbE?usp=sharing. Then place "best_prior.pt" into VQVAE/VQVAE/models_pretrained/ and place "wide_resnet50_2.pt" into VQVAE/classifier/models. To evaluate the pre-trained models, run:

run_vqvae_pretrained.sh

For the behavior prediction experiments, please see the README.md in the "Behavior Prediction/" folder. Example results on the behavior prediction task are shown below (Figure 4 in the paper):

alt text

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published