Skip to content
No description, website, or topics provided.
Python Shell sed
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

ICML-2019 Supporting Code for Submission #960 (Model Comparison for Semantic Grouping)

This repo consits of a slight set of modifications for SentEval to reproduce results in accepted submission #960 @ICML2019. Manuscript can be found here.

NOTE: This repo is a fork of SentEval. Any commits prior to 2019 are not associated to publication #960 @ICML2019 .


This code is written in python. The dependencies are:

Download datasets

To get all the transfer tasks datasets, run (in data/) using Bash >= 4.0:


This will automatically download and preprocess the datasets, and store them in data/senteval_data (warning: for MacOS users, you may have to use p7zip instead of unzip). Note: we provide PTB or MOSES tokenization.

WARNING: Extracting the MRPC MSI file requires the "cabextract" command line (i.e apt-get/yum install cabextract).

This will also download glove.840B.300d.txt and enwiki_vocab_min200.txt (The SIF frequencies from Arora et al. 2016).

To download the other word vectors please go to GoogleNews-word2vec and FastText, then convert binary files into the same text file format as glove.840B.300d.txt and place in /data/word_vectors. We could not upload them to GitHub since they are above the allowed disk-quota.

Reproduce Results for Submission #960

In order to reproduce results please run:

cd examples
python # To reproduce Arora et al. (2016)'s SIF+PCA results
python # To reproduce our Gaussian-AIC/TIC results
python  # To reproduce our vMF-AIC/TIC results

This will reproduce the results for glove.840B.300d.txt and potentially crash afterwards if you have not downloaded the other word vectors.

Similarity code

The entire codebase connected to the similarity metrics described in the paper is encapsulated in the similarity folder. This is where the core contributions of our work are.

You can’t perform that action at this time.