FEVER Shared Task 2018
The First Workshop on Fact Extraction and Verification
To reproduce our FEVER shared task results:
- switch to the takuma-dev branch
bash initial_setup.sh(This will download several files and take some time.)
- move to
jackdirectory and install dependencies according to the README. (i.e.,
python3 -m pip install -e .[tf])
- move to
fever-baselinesdirectory and install dependencies (i.e.,
pip install -r requirements.txt)
After step 2,
fever-baselines directory should be at the same level (these should be in the same directory).
Reproduce our result
python3 pipeline.py --config configs/submission_config.json --model [arbitrary name]
Output files will be generated under
test_submission.json correspond to the output for development and test set for each.
(Note that the score displayed after running this particular model is not valid, since we included development set for the training even though the score is calculated based on the development set.)
Train a model with new data
Use the same wiki data
- create new configuration file
configs/config.jsonand reflect your directory structure
python3 pipeline.py --config configs/config.json --model [arbitrary name]
Use a new/different wiki-pages data
- remove index files in
python3 doc_ir_model.pyto create a document index and retrieval model
python3 line_ir_model.pyto create a line index and retrieval model
Configuration files can have a parent, which is specified by
Attributes that are not specified in the child config file are inherited from the parent. Usually you do not need to modify the parent config.
Note that currently a child cannot have its child. (Having a grand-parent or grand-child is not supported)
Information Retrieval module.
Data format conversion module (to let jack handle our data).
Recognizing Textual Entailment module (training).
Recognizing Textual Entailment module (inference).