SCDE: Sentence Cloze Dataset with High Quality Distractors From Examinations

Code for the paper:

Sentence Cloze Dataset with High Quality Distractors From Examinations. Xiang Kong*, Varun Gangal*, and Eduard Hovy. ACL2020.

Leaderboard

If you have new results, it would be great if you could submit it here (https://paperswithcode.com/sota/question-answering-on-scde).

Dependencies

• Python 3.6+
• Pytorch 1.2
• Transformers 2.1.1

Datasets

• SCDE: Please submit a data request here. The data will be automatically sent to you. Please also check your spam folder.

Usage

Installing the Transformers from the source

cd transformers
pip install .

Preprocessing (get the AP+AN context features)

python extract_features.py --output_dir all_prev_next_test --input_dir scde_data/ --feature_type apn

Finetune a BERT-based model in folder transformers

bash train.sh feature_dir

Reference

If you find our data or code useful, please consider citing our paper:

@inproceedings{xiang2020sentence,
  title={SCDE: Sentence Cloze Dataset with High Quality Distractors from Examinations},
  author={Kong, Xiang and Gangal, Varun and Hovy, Eduard},
  booktitle={Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year={2020}
}

Acknowledgement

• The code is adapted from Transformers (https://github.com/huggingface/transformers). Thanks!

License

MIT