Please install python 3.8 and pytorch 1.6. You'll also need
- ax
- huggingface transformers
- numpy
- scikit-learn
In order to run LIME explanation generation, you will also need
- lime
The main entry point for this code is main.py
. You can run training as follows:
python main.py train --main_dataset path/to/train/fn --dev_file path/to/dev/fn --test_file path/to/test/fn --model model_name
The train, development, and test files should all be .csv files with two columns: text and then labels.
Various additional train flags and arguments may be helpful:
--optimize
, which will use theax
library to optimize parameters.--save_path
, for saving the model. The code will append-params.pth
onto this to save the model, and will also save meta-information, like the label2idx associations and test scores, with-meta.pkl
.--save_lm
, which will save the language model (BertModel
) only. This requires--save-path
and will append-lm
to the path. This is useful for sequential training scripts, such as our Fine-Tune models.--log
, to save the log to a file as well as print it to stdout, and--logfile
, to specify the log location.
Multitask learning is supported by the --aux_datasets
argument, which should be given a list of files in the same
format as the train file.
TODO: add training commands in bin/
after results are finalized.
You can run prediction using a saved model as follows:
python main.py predict --model_path path/to/model --data path/to/data/to/label --out_file path/to/save/data
The data to label must be in the same format as the training data (two columns, with headers).
Provide the --model_path
in the same format as the --save_path
for training, without any -params.pth
suffix,
because the code will need to load both the model parameters and the pickled meta-information.
You can run analysis from the paper using interpret_lime.py
and then interpret_liwc.py
. These scripts expect certain
models to be saved in a saved_models/
directory; you may change these model names in the relevant loops. Create a
lime/
directory in the top level of this repository before running either.
interpert_liwc.py
also expects a LIWC word list to be present as data/liwc.tsv
. We cannot share our copy of this
file, but it was created using LIWC 2015 and is in .tsv
format, with one column 'Category' and another 'Words'.
'Words' is a space-delimited list of strings from LIWC, where an asterisk () is included in some words to denote that
any ending is accepted (e.g., "walk" would accept "walking", "walks", and "walked" as well as "walk").