📋 End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models

End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models

This repository is the official implementation of End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models. Best Paper Honorable Mention at SIGIR 2023.

Please use the following citation:

@inproceedings{10.1145/3539618.3591879,
author = {Yao, Barry Menglong and Shah, Aditya and Sun, Lichao and Cho, Jin-Hee and Huang, Lifu},
title = {End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models},
year = {2023},
isbn = {9781450394086},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3539618.3591879},
doi = {10.1145/3539618.3591879},
booktitle = {Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval},
pages = {2733–2743},
numpages = {11},
keywords = {explainable fact-checking, multimodal fact-checking, explanation generation, evidence retrieval, stance detection},
location = {Taipei, Taiwan},
series = {SIGIR '23}
}

Requirements

To install requirements:

conda create -n mocheg python=3.8.10
conda install pytorch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=10.2 -c pytorch
pip install -r requirements.txt

Dataset - MOCHEG

You can download dataset here:

• MOCHEG version 1.

• Dataset Format and structure are explained in document/MOCHEG_dataset_statement.pdf.

Pre-trained Models

You can download pretrained models here:

• pretrained models trained on MOCHEG.

Evaluation

To evaluate models on MOCHEG, run:

bash eval.sh

---

Training (Optional)

If you want to train models by yourself, run this command:

bash train.sh

Dataset Build (Optional)

If you want to build dataset by yourself, run this command:

bash build_dataseth.sh

Credit:

This repository was build by Barry Menglong Yao, Aditya Shah.

The data crawler scripts are based on conll2019-snopes-crawling.

Contributing

📋 Our dataset is licensed under the CC BY 4.0. The associated codes are licensed under Apache License 2.0.