Skip to content
Repository for deep zero-shot learning via cross-modality transfer
Branch: master
Clone or download
Latest commit ff09101 Apr 25, 2014
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
attributeTraining Finalize README Apr 25, 2014
buildFeatures
costFunctions Add ability to reduce change in certain weights. May 29, 2013
evaluateFunctions
figures Clean up filenames for plotting and evaluation functions Apr 25, 2014
gaussianFunctions Pare size of main folder by removing old/unused scripts Apr 25, 2014
loopFunctions Pare size of main folder by removing old/unused scripts Apr 25, 2014
plotting Finalize README Apr 25, 2014
toolbox Add remaining MATLAB files Apr 15, 2014
trainFunctions Pare size of main folder by removing old/unused scripts Apr 25, 2014
tsne Add evaluation portion to map-based training Dec 1, 2012
word_data/acl Add ignored wordTable files for CIFAR Apr 25, 2014
.gitignore
README.md Finalize README Apr 25, 2014
evaluateGaussianBayesian.m Clean up filenames for plotting and evaluation functions Apr 25, 2014
evaluateLoopBayesian.m Clean up filenames for plotting and evaluation functions Apr 25, 2014
generateGaussianCutoffs.m Clean up filenames for plotting and evaluation functions Apr 25, 2014
initializeParameters.m
loadData.m Finalize README Apr 25, 2014
main.m Clean up filenames for plotting and evaluation functions Apr 25, 2014
mapDoMap.m

README.md

Zero-shot learning via cross-modality transfer

This package contains the code for the paper Zero-Shot Learning Through Cross-Modal Transfer presented at NIPS 2013.

Please cite the code as follows:

Zero-Shot Learning Through Cross-Modal Transfer. Richard Socher, Milind Ganjoo, Christopher D. Manning, Andrew Y. Ng. Advances in Neural Information Processing Systems (NIPS 2013).

Getting started

To run our model and generate some of the figures presented in the paper, follow the following steps.

Preparing image data

  1. Download MATLAB version of dataset from http://www.cs.toronto.edu/~kriz/cifar.html.
  2. Merge matrices from different batches into one file -- train.mat -- and rename the test batch to test.mat and the meta file to meta.mat.
  3. Create folder image_data and its subdirectories, images/ and features/.
  4. Move all three files to image_data/images/cifar10.
  5. Run buildFeatures.m in buildFeatures/ to create image features for use in our model. The script will create train.mat and test.mat in image_data/features/cifar10. [1]

Preparing word vectors

We primarily used word vectors from [2], extracting 10 relevant vectors for CIFAR-10 and 96 vectors for CIFAR-100 (vectors for 4 categories from CIFAR-100 were not present in the vocabulary). You may use these directly. You can also use a different source of word vectors, saving the extracted word table for the CIFAR-10 classes under word_data/(dataset_name)/cifar10/wordTable.mat

Running model

The model can be run by executing main.m in the root directory, which will tune model parameters and generate a few graphs.

References

[1] The Importance of Encoding Versus Training with Sparse Coding and Vector Quantization. A. Coates and A. Ng. In ICML, 2011.

[2] Improving Word Representations via Global Context and Multiple Word Prototypes. Eric H. Huang, Richard Socher, Christopher D. Manning and Andrew Y. Ng. Association for Computational Linguistics 2012 Conference (ACL 2012).

You can’t perform that action at this time.