Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Word Sense Induction with Neural biLM and Symmetric Patterns

This repository contains reproducing code for the results in the paper:

Word Sense Induction with Neural biLM and Symmetric Patterns

Usage instructions:

The code is written in python 3.6 and runs the SemEval 2013 task 13 evaluation code provided by the SemEval 2013 workshop, which is written in Java and therefore require installed JRE.

It also requires the AllenNLP python package as well as a few other packages. Below are detailed instructions for running the code.


  • Java >=1.6 installed and in PATH
  • Python 3.6, preferably a clean Conda environment

Installing a clean conda environment from scratch:

# create and activate new python 3.6 conda envrionment
conda create -n spwsi python=3.6 anaconda
source activate spwsi

# for cuda support install pytorch first (see link below and note the cuda version!)
# in our system a full run w/ cuda took ~2 minutes. w/o cuda ~20 minutes
# if you don't mind that,
# you can skip next line and later run with arguments: --cuda -1
conda install pytorch cuda80 -c pytorch

# additional python pacakges from requirements.txt
pip install -r requirements.txt

# install spacy English model, download ELMo's output matrix and SemEval 2013 task code
python -m spacy download en

for pytorch installation w/ cuda support see instructions at pytorch website

finally, to do the actual WSI using ELMo's LM, run Arguments of listed below. Note that the script is creating a lemmatized vocabulary file in the resources directory and therefore needs write permission to it

we also provide to run a batch of runs, such as ablation or paramters search, while keeping relevant data structures in memory. script arguments:

python -h
usage: [-h] [--n-clusters N_CLUSTERS]
                     [--n-representatives N_REPRESENT]
                     [--n-samples-side N_SAMPLES_SIDE] [--cuda CUDA]
                     [--debug-dir DEBUG_DIR] [--disable-lemmatization]
                     [--disable-symmetric-patterns] [--disable-tfidf]
                     [--append-results-file APPEND_RESULTS_FILE]
                     [--run-prefix RUN_PREFIX]
                     [--elmo-batch-size ELMO_BATCH_SIZE]
                     [--prediction-cutoff PREDICTION_CUTOFF]
                     [--cutoff-elmo-vocab CUTOFF_ELMO_VOCAB]

BiLM Symmetric Patterns WSI Demo

optional arguments:
  -h, --help            show this help message and exit
  --n-clusters N_CLUSTERS
                        number of clusters per instance (default: 7)
  --n-representatives N_REPRESENT
                        number of representations per sentence (default: 20)
  --n-samples-side N_SAMPLES_SIDE
                        number of samples per representations side (default:
  --cuda_device CUDA_DEVICE
                        cuda device for ELMo (-1 to disable) (default: 0)
  --debug-dir DEBUG_DIR
                        logs and keys are written will be written to this dir
                        (default: debug)
                        disable ELMO prediction lemmatization (default: False)
                        disable "x and y" symmetric pattern and predict
                        substitutes inplace (default: False)
  --disable-tfidf       disable tfidf transformer (default: False)
  --run-postfix RUN_POSTFIX
                        will be appended to log file names and products
                        (default: )
  --lm-batch-size LM_BATCH_SIZE
                        ELMo prediction batch size (optimization only)
                        (default: 50)
  --prediction-cutoff PREDICTION_CUTOFF
                        ELMo predicted distribution top K cutoff (default: 50)
  --cutoff-lm-vocab CUTOFF_LM_VOCAB
                        optimization: only use top K words for faster output
                        matrix multiplication (default: 50000)

The default parameters specified above are used in the paper

to run on cuda device 1

python --cuda_device 1

After running, a detailed execution log will be available inside "debug" directory after running the script.

Additionally, a SemEval .key file will be created inside the "debug" dir. This file will contain a mapping from the task dataset entries to their induced senses. The SemEval task code evaluates this mapping and produce two metric scores: FNMI and FBC as described in the paper. You can re-run this key file evaluation using

Developed and tested on Ubuntu 16, Cuda 8.0

Usage by filename:

  • - download SemEval 2013 task dataset and ELMo's output matrix weights and put them in the resource directory

  • - induce senses from SemEval 2013 task dataset

  • - run a batch of induction runs efficiently, provided scenarios: ablation, search, n_clusters. edit code and run for modifications

  • - after each run, a SemEval .key file is produced in the debug dir, you can re-run and print task scoring on the .key file produced using this script

  • spwsi/ - if you want to use a custom biLM, this interface should be implemented


As detailed in the paper, the proposed method achieves these scores on SemEval 2013 task 13: Score: mean(standard deviation) FNMI: 11.26(0.43) FBC: 57.49(0.23) AVG: 25.43(0.48)


Word Sense Induction with neural Bi-language Models and symmetric patterns







No releases published


No packages published