Skip to content

stanfordnlp/coqa

Repository files navigation

CoQA -- A Conversational Question Answering Challenge

The goal of CoQA challenge is to measure the ability of machines to read a passage and answer a series of interconnected questions that appear in a conversation. This repository hosts the code for generating the website and the associated data files. The resulting web files are stored on the gh-pages branch.

License

CoQA contains data from seven domains. Of these, only five are shared for public and the remaining two are private (for out-of-domain testing).

  • Literature and Wikipedia domains are shared under CC BY-SA 4.0 license.
  • Children's stories are collected from MCTest.
  • Mid/High school exam passages are collected from RACE
  • News articles are collected from CNN dataset.

Commands

To automate sanity checks, run the following commands

# First create submissions/FILE.txt indicating the worksheet, cl run command, submission name and date, e.g. see submissions/BERT-Augmented-AoA.txt
python print-coqa-eval-commands.py < submissions/FILE.txt > submissions/FILE.sh
sh submissions/FILE.sh 

To see if the submission succeeded, run the following command

python populate_results.py < submissions/FILE.txt

To populate the leaderaboard, run the following command

cp out-v1.0.new.json out-v1.0.json
gulp
gulp connect
gulp deploy

About

CoQA -- A Conversational Question Answering Challenge

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published