Skip to content

CogComp/TacoLM

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 

TacoLM

TemporAl COmmon Sense Language Model

A variation of BERT that is aware of temporal common sense.

Introduction

This is the code repository for our ACL 2020 paper Temporal Common Sense Acquisition with Minimal Supervision. This package is built upon huggingface/transformers at its April 2019 version.

Installation

  • pip install -r requirements.txt
  • pip install --editable .

Out of the box

Here are some things you can do with this package out of the box.

Train the main model

  • Access and download data/tmp_seq_data at Google Drive (4.6 G)
  • run sh train_taco_lm.sh

The script is set to default parameters and will export the model to models/. You can configure differently by editing the script.

The training process will generate one directory to store the loss logs, as well as NUM_EPOCH directories for each epoch's model. You will need to add BERT's vocab.txt to the epoch directories for evaluation. See more detail in the next section on pre-trained models.

The training data is pre-generated and formatted. More details here.

Experiments

You can download pre-trained models in models/ at Google Drive (0.4 G each), or follow the training procedure in the previous section.

General Usage

You can do many things with the model by just treating it as a set of transformer weights that fit exactly into a BERT-base model. Have an on-going project with BERT? Give it a try!

Intrinsic Experiments

The intrinsic evaluation relies on pre-formatted data.

  • run sh eval_intrinsic.sh
  • see eval_results/intrinsic.txt for results

TimeBank Experiment

  • by default this requires the epoch 2 model.
  • run sh eval_timebank.sh to produce evaluation results on 3 different seeds. They are by default stored under eval_results
  • run python scripts/eval_timebank.py to see result interpretations.

HiEVE Experiment

  • by default this requires the epoch 2 model.
  • run sh eval_hieve.sh to produce eval results under eval_results
  • run python scripts/eval_hieve.py to see interpretations.

MC-TACO Experiment

See MC-TACO.

  • use the augmented data under data/mctaco-tcs
  • use the transformer weights of taco_lm_epoch_2

Citation

See the following paper:

@inproceedings{ZNKR20,
    author = {Ben Zhou, Qiang Ning, Daniel Khashabi and Dan Roth},
    title = {Temporal Common Sense Acquisition with Minimal Supervision},
    booktitle = {ACL},
    year = {2020},
}

About

Temporal Common Sense Acquisition with Minimal Supervision, ACL'20

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published