Skip to content
Iterative Rank-Aware Open IE
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data add test conll file Jun 24, 2019
pretrain/vocabulary init commit May 30, 2019
rerank init commit May 30, 2019
training_config init commit May 30, 2019
README.md modify paper link Jun 3, 2019
init_iter.sh
iter.sh update installation & readme May 31, 2019
openie_extract.py modify script location & modify readme May 30, 2019
rerank.py modify script location & modify readme May 30, 2019
rerank_to_oie.sh modify script location & modify readme May 30, 2019

README.md

Iterative Rank-Aware Open IE

This repository contains data and code to reproduce the experiments in:

Improving Open Information Extraction via Iterative Rank-Aware Learning

by Zhengbao Jiang, Pengcheng Yin and Graham Neubig.

Install

Our model implementation is based on AllenNLP, and evaluation is based on supervised-oie.

  1. Prepare the model repository:

    git clone https://github.com/jzbjyb/oie_rank.git
    cd oie_rank
    conda create -n oie_rank python=3.6
    conda activate oie_rank
    pip install allennlp==0.8.2
    conda deactivate
    cd ..
  2. Prepare the evaluation repository:

    git clone https://github.com/jzbjyb/supervised-oie.git
    cd supervised-oie
    # rollback to the evaluation metrics used in this paper
    git reset --hard ef66de761a5230a7905918b4c7ce87147369bf33
    conda create -n sup_oie python=2.7
    conda activate sup_oie
    pip install docopt==0.6.2 numpy==1.11.2 scikit-learn==0.18 scipy==0.18.1 xtermcolor==1.3 termcolor==1.1.0 nltk==3.2.5 pandas==0.19.0 requests==2.21.0
    conda deactivate
    cd ..

Download pre-trained models

  1. Download pre-trained ELMo (weights and options).

    cd oie_rank
    mkdir pretrain/elmo/
    cd pretrain/elmo/
    wget https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway_5.5B/elmo_2x4096_512_2048cnn_2xhighway_5.5B_weights.hdf5
    wget https://s3-us-west-2.amazonaws.com/allennlp/models/elmo/2x4096_512_2048cnn_2xhighway_5.5B/elmo_2x4096_512_2048cnn_2xhighway_5.5B_options.json
  2. Download pre-trained RnnOIE (base model in our paper) and its extractions. gdown.pl can be download from here.

    cd oie_rank
    mkdir pretrain/rnnoie/
    cd pretrain/rnnoie/
    # download model
    gdown.pl https://drive.google.com/open?id=1ykuxnIE1vkhJeq-81UPtAs6K6_h3Vcw3 model.tar.gz
    # download extractions
    gdown.pl https://drive.google.com/open?id=1Mn2ptAAiB5rTRg3GYMtBk9lrFcC3Qb4R oie2016.txt

Iterative rank-aware learning

First, create a directory to hold data, models, and extractions during iterative training.

./init_iter.sh ROOT_DIR

Then, run iterative rank-aware training (3 iterations with beam size of 5).

CONDA_HOME=PATH_TO_YOUR_CONDA_HOME ./iter.sh 3 ROOT_DIR training_config/iter.jsonnet 5

The extractions generated at i-th iteration are saved to ROOT_DIR/eval/iteri/tag_model/. The reranking results of extractions generated at i-1-th iteration are saved to ROOT_DIR/eval/iteri/. Code for rank-aware binary classification loss is in rerank/models/reranker.py, which is really simple and could be easily applied to other models and tasks.

Train and evaluate RnnOIE

In addition to using pretrained RnnOIE model, you can also use the following commands to train from scratch and evaluate.

cd oie_rank
conda activate oie_rank
# train
allennlp train training_config/oie.jsonnet \
    --include-package rerank \
    --serialization-dir MODEL_DIR
# generate extractions
python openie_extract.py \
    --model MODEL_DIR/model.tar.gz \
    --inp data/sent/oie2016_test.txt \
    --out RESULT_DIR/oie2016.txt \
    --keep_one
# evaluate
cd ../supervised-oie/supervised-oie-benchmark/
conda activate sup_oie
python benchmark.py \
    --gold oie_corpus/test.oie.orig.correct.head \
    --out /dev/null \
    --tabbed RESULT_DIR/oie2016.txt \
    --predArgHeadMatch

Reference

@inproceedings{jiang19acl,
    title = {Improving Open Information Extraction via Iterative Rank-Aware Learning},
    author = {Zhengbao Jiang and Pengcheng Yin and Graham Neubig},
    booktitle = {The 57th Annual Meeting of the Association for Computational Linguistics (ACL)},
    address = {Florence, Italy},
    month = {July},
    year = {2019}
}
You can’t perform that action at this time.