Linguistically-Informed Self-Attention implemented in TensorFlow
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin print CUDA_VISIBLE_DEVICES in eval script Dec 16, 2018
config update fnames Dec 16, 2018
src undo checkpoints Dec 16, 2018
LICENSE add license Sep 23, 2018
lisa.jpg readme Jul 16, 2018

LISA: Linguistically-Informed Self-Attention

This is a work-in-progress, but much-improved, re-implementation of the linguistically-informed self-attention (LISA) model described in the following paper:

Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. Linguistically-Informed Self-Attention for Semantic Role Labeling. Conference on Empirical Methods in Natural Language Processing (EMNLP). Brussels, Belgium. October 2018.

To exactly replicate the results in the paper at the cost of an unpleasantly hacky codebase, you can use the original LISA code here.


  • >= Python 3.6
  • >= TensorFlow 1.10

Quick start:

Data setup (CoNLL-2005):

  1. Get pre-trained word embeddings (GloVe):
    wget -P embeddings
    unzip -j embeddings/ glove.6B.100d.txt -d embeddings
  2. Get CoNLL-2005 data in the right format using this repo. Follow the instructions all the way through further preprocessing.
  3. Make sure the correct data paths are set in config/conll05.conf

Train a model:

To train a model with save directory model using the configuration conll05-lisa.conf:

bin/ config/conll05-lisa.conf --save_dir model

Evaluate a model:

To evaluate the latest checkpoint saved in the directory model:

bin/ config/conll05-lisa.conf --save_dir model