MUSE

This repository provides the data and code to reproduce the results of Correcting misinformation on social media with a large language model.

Instructions

• data/notes_all.csv: The Community Notes data that our evaluation is based on.

• data/responses.csv: It contains the tweets and responses made by

  1. Laypeople with high helpfulness;
  2. Laypeople with average helpfulness;
  3. MUSE that simulates correcting misinformation at the same time as the laypeople's high-helpfulness response;
  4. MUSE that simulates correcting misinformation at the same time as the laypeople's average-helpfulness response;
  5. MUSE that simulates correcting misinformation right after it appears on social media;
  6. MUSE\retrieval (multimodal inputs only, otherwise it is the same as GPT-4);
  7. MUSE\vision (multimodal inputs only, otherwise it is the same as GPT-4); and
  8. GPT-4.

  '~' indicates the response is the same as (iii). '|||': the same as (iv). '*': the same as (vi). '$': the same as (vii). '///': the same as (viii).

• data/Q[..].csv: It contains the experts' evaluation results of the responses in data/responses.csv.

• data/username_tweetids.csv: The assignment of the tweets and responses to every expert in the annotation phase.

• data/tweetid_domain: The identified domain of each tweet.

• code/: The code to reproduce the main results in our paper. The results were generated with Python 3.7 and dependencies in requirements.txt.

Notes:

• We comply with X/Twitter Terms of Service by only releasing the IDs of tweets. Most code files are runnable without further obtaining the tweet data, except fig_s17.ipynb, fig_s27.ipynb, and fig_s28.ipynb, where the creation times of tweets are necessary.
• The names of the experts are anonymized.

Citation

@article{zhou2024muse,
  title={Correcting misinformation on social media with a large language model},
  author={Zhou, Xinyi and Sharma, Ashish and Zhang, Amy X and Althoff, Tim},
  journal={arXiv preprint arXiv:2403.11169},
  year={2024}
}