LISA: Linguistically-Informed Self-Attention
This is a work-in-progress, but much-improved, re-implementation of the linguistically-informed self-attention (LISA) model described in the following paper:
Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. Linguistically-Informed Self-Attention for Semantic Role Labeling. Conference on Empirical Methods in Natural Language Processing (EMNLP). Brussels, Belgium. October 2018.
To exactly replicate the results in the paper at the cost of an unpleasantly hacky codebase, you can use the original LISA code here.
- >= Python 3.6
- >= TensorFlow 1.10
Data setup (CoNLL-2005):
- Get pre-trained word embeddings (GloVe):
wget -P embeddings http://nlp.stanford.edu/data/glove.6B.zip unzip -j embeddings/glove.6B.zip glove.6B.100d.txt -d embeddings
- Get CoNLL-2005 data in the right format using this repo. Follow the instructions all the way through further preprocessing.
- Make sure the correct data paths are set in
Train a model:
To train a model with save directory
model using the configuration
bin/train.sh config/conll05-lisa.conf --save_dir model
Evaluate a model:
To evaluate the latest checkpoint saved in the directory
bin/evaluate.sh config/conll05-lisa.conf --save_dir model