Skip to content
Dataset and code for “Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding, EMNLP 2019.
Python Other
  1. Python 98.8%
  2. Other 1.2%
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
dataset
evaluator
experiments
README.md
_config.yml

README.md

MC-TACO 🌮

Dataset and code for “Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding EMNLP 2019. (link)

Dataset

We provide the dev/test split as specified in the paper, along with a detailed readme.txt file under dataset/

Leaderboard

See the details and instructions at: http://leaderboard.allenai.org/mctaco

Experiments (WIP)

At this point, we provide the outputs of the ESIM/BERT baselines.

To run BERT baseline:

First install required packages with:

pip install -r experiments/bert/requirements.txt

Your the following command to reproduce BERT predictions under ./bert_output:

sh experiments/bert/run_bert_baseline.sh

Evaluate the predictions with which you can further evaluate with the following command:

python evaluator/evaluator.py eval --test_file dataset/test_9442.tsv --prediction_file bert_output/eval_outputs.txt

ESIM baseline: Releasing soon after some polish

Citation

See the following paper:

@inproceedings{ZKNR19,
    author = {Ben Zhou, Daniel Khashabi, Qiang Ning and Dan Roth},
    title = {“Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding },
    booktitle = {EMNLP},
    year = {2019},
}
You can’t perform that action at this time.