TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
- This repo contains code for the paper Mandar Joshi, Eunsol Choi, Daniel Weld, Luke Zettlemoyer.
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension In Association for Computational Linguistics (ACL) 2017, Vancouver, Canada.
- The data can be downloaded from the TriviaQA website.
- Please contact Mandar Joshi (<first-name>email@example.com) for suggestions and comments.
- Python 3. You should be able to run the evaluation scripts using Python 2.7 if you take care of unicode in
- BiDAF requires Python 3 -- check the original repository for more details.
- tensorflow (only if you want to run BiDAF, verified on r0.11)
dataset file parameter refers to files in the
qa directory of the data (e.g.,
wikipedia-dev.json). For file format, check out the
sample directory in the repo.
python3 -m evaluation.triviaqa_evaluation --dataset_file samples/triviaqa_sample.json --prediction_file samples/sample_predictions.json
- If you have a SQuAD model and want to run on TriviaQA, please refer to