Skip to content
Fine-tuning GPT-2 Small for Question Answering
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker
gpt2sqa init Jun 1, 2019
LICENSE init Jun 1, 2019
MANIFEST.in init Jun 1, 2019
README.md Update README.md May 30, 2019
evaluate-v2.0.py init Jun 1, 2019
gpt2_squad.py init Jun 1, 2019
requirements.txt init Jun 1, 2019
setup.py init Jun 1, 2019

README.md

GPT2sQA

This repo includes an experiment of fine-tuning GPT-2 117M for Question Answering (QA). It also runs the model on Stanford Question Answering Dataset 2.0 (SQuAD). It uses Huggingface Inc.'s PyTorch implementation of GPT-2 and adapts from their fine-tuning of BERT for QA.

SQuAD data can be downloaded from: https://github.com/rajpurkar/SQuAD-explorer/tree/master/dataset

To train and validate the model:

python gpt2_squad.py --output_dir=output/ --train_file=data/train-v2.0.json --do_train --train_batch_size=32 --predict_file=data/dev-v2.0.json --do_predict

To evaluate:


python evaluate-v2.0.py data/dev-v2.0.json output/predictions.json

Different fine-tuning experiments will be uploaded soon for GPT-2 345M on datasets that exclusively target commonsense reasoning in an attempt to bring insight to reasoning abilities of GPT-2. Such an insight could potentially improve our ability to improve Natural Language Understanding through language models in semi-supervised settings.

You can’t perform that action at this time.