BAG

Implementation for NAACL-2019 paper

BAG: Bi-directional Attention Entity Graph Convolutional Network forMulti-hop Reasoning Question Answering

paper link

Requirement

We provided main entrance in both TensorFlow version (BAG.py) and PyTorch version (BAG-pytorch.py)

1. Python 3.6
2. TensorFlow == 1.11.0 (if you want to run TF version script, We are not sure if it works at higher version)
3. Pytorch >= 1.1.0
4. SpaCy >= 2.0.12 (You need to install "en" module via "python -m spacy download en")
5. allennlp >= 0.7.1
6. nltk >= 3.3
7. pytorch-ignite (if your need to run the script in PyTorch version)

And some other packages

I run it using two NVIDIA GTX1080Ti GPUs each one has 11GB memory. To run it with default batch size 32, at least 16GB GPU memory is needed. To run the preprocessing procedure on the whole dataset, at least 50GB system memory is needed.

How to run

• Before run

You need to download pretrained 840B 300d GLoVe embeddings, and pretrained original size ELMo embedding weights and options and put them under directory /data.

• Preprocessing dataset

You need to download QAngaroo WIKIHOP dataset , unzip it and put the json files under the root directory. Then run prerpocessing script

python prepro.py {json_file_name}

It will generate four preprocessed pickle file in the root directory which will be used in the training or prediction.

• Train the model

Train the model using following command which will follow the configure in original paper

python BAG.py {train_json_file_name} {dev_json_file_name} --use_multi_gpu=true

or in pytorch version (we do not provide multi-gpu support yet for pytorch, the most simple way is wrapping model with nn.DataParallel)

python BAG-pytorch.py {train_json_file_name} {dev_json_file_name}

Please make sure you have run preprocessing for both train file and dev file before training. And please make sure you have CUDA0 and CUDA1 available. If you have single GPU with more than 16GB memory, you can remove parameter --use_multi_gpu.

• Predict

After training it will put trained model onto directory /models. You can predict the answer of a json file using following command

python BAG.py {predict_json_file_name} {predict_json_file_name} --use_multi_gpu=true --evaluation_mode=true

• Trained model

Anyone who needs the trained model in our submission can find it on the Codalab (Only TF version is available)

Acknowledgement

We would like to appreciate Nicola De Cao link for his assistance in implementing this project.

Reference

@inproceedings{cao2019bag,
  title={BAG: Bi-directional Attention Entity Graph Convolutional Network for Multi-hop Reasoning Question Answering},
  author={Cao, Yu and Fang, Meng and Tao, Dacheng},
  booktitle={Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  pages={357--362},
  year={2019}
}