The Many Dimensions of Truthfulness: Crowdsourcing Misinformation Assessments on a Multidimensional Scale
To Appear.
This repository contains the crowdsourced judgments used in the PAUC paper titled "Can the Crowd Judge Truthfulness? A Longitudinal Study on Recent Misinformation about COVID-19"
If you use this resource, please cite our paper:
Kevin Roitero, Michael Soprano, Beatrice Portelli, Massimiliano De Luise, Damiano Spina, Vincenzo Della Mea, Giuseppe Serra, Stefano Mizzaro, and Gianluca Demartini. Can the Crowd Judge Truthfulness? A Longitudinal Study on Recent Misinformation about COVID-19. In Personal and Ubiquitous Computing, 2021.
@article{roitero2020infodemic,
author = {Roitero, Kevin and Soprano, Michael and Portelli, Beatrice and De Luise, Massimiliano and Spina, Damiano and
Della Mea, Vincenzo and Serra, Giuseppe and Mizzaro, Stefano and Demartini, Gianluca},
title = {Can the Crowd Judge Truthfulness? A Longitudinal Study on Recent Misinformation about COVID-19},
booktitle = {Personal and Ubiquitous Computing},
year={2021}
}This repository contains the crowdsourced judgments used in the CIKM'20 full paper titled "The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?"
If you use this resource, please cite our paper:
Kevin Roitero, Michael Soprano, Beatrice Portelli, Damiano Spina, Vincenzo Della Mea, Giuseppe Serra, Stefano Mizzaro and Gianluca Demartini. 2020. The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?. In CIKM ’20: 29th ACM International Conference on Information and Knowledge Management (CIKM2020), Online. October 19-23, 2020.
@InProceedings{roitero2021longitudinal,
author = {Roitero, Kevin and Soprano, Michael and Portelli, Beatrice and Spina, Damiano and
Della Mea, Vincenzo and Serra, Giuseppe and Mizzaro, Stefano and Demartini, Gianluca},
title = {The COVID-19 Infodemic: Can the Crowd Judge Recent Misinformation Objectively?},
booktitle = {Proceedings of CIKM'20},
year={2020}
}Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background
This repository contains the crowdsourced judgments used in the SIGIR'20 full paper titled "Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor's Background"
If you want to see our talk at SIGIR 2020 visit this link: SIGIR 2020 Talk Video
If you use this resource, please cite our paper:
Kevin Roitero, Michael Soprano, Shaoyang Fan, Damiano Spina, Stefano Mizzaro, and Gianluca Demartini. 2020. Can The Crowd Identify Misinformation Objectively? The Effects of Judgment Scale and Assessor’s Background.In SIGIR ’20: The 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 25–30, 2020, Xi’an, China. ACM.
@InProceedings{roitero2020can,
author = {Roitero, Kevin and Soprano, Michael and Fan, Shaoyang and Spina, Damiano
and Mizzaro, Stefano and Demartini, Gianluca},
title = {Can The Crowd Identify Misinformation Objectively?
The Effects of Judgment Scale and Assessor’s Background},
booktitle = {Proceedings of SIGIR'20},
year={2020}
}This work is partially supported by a Facebook Research award and by an Australian Research Council Discovery Project (DP190102141). We thank Devi Mallal from RMIT ABC Fact Check for facilitating access to the ABC dataset.
This repository contains the crowdsourced judgments used in the ECIR'20 short paper titled "Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias", for two two different judgment scales: S6 and S100.
The files also include the collected information about assessors’ background that allowed us to analyse assessment bias.
If you use this resource, please cite our paper:
La Barbera D., Roitero K., Demartini G., Mizzaro S., Spina D. (2020) Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias. In: Jose J. et al. (eds) Advances in Information Retrieval. ECIR 2020. Lecture Notes in Computer Science, vol 12036. Springer, Cham. Best Short Paper Award.
@inproceedings{labarbera2020crowdsourcing,
title={{Crowdsourcing Truthfulness: The Impact of Judgment Scale and Assessor Bias}},
booktitle={Proceedings of ECIR'20},
author={{La Barbera}, David and Roitero, Kevin and Demartini, Gianluca and Mizzaro, Stefano and Spina, Damiano},
year={2020}
}- Presentation: http://www.youtube.com/watch?v=9wFFMcplvjk
- Paper: https://link.springer.com/chapter/10.1007/978-3-030-45442-5_26
This work is partially supported by an Australian Research Council Discovery Project (DP190102141) and a Facebook Research award.