Skip to content
Permalink
master
Switch branches/tags
Go to file
 
 
Cannot retrieve contributors at this time

Studies and tools using lists

Researchers rely on existling lists of unreliable-news sites for their studies. But these lists are often out-of-date and incomplete, so their research may lead to flawed conclusions.

Here's how that error could easily happen: Two frequently used lists are PolitiFact's Fake News Almanac and OpenSources. Both haven't been updated since 2017. Say you used these lists to compare social media shares from fake-news site articles in 2016 with those in 2018. You count twice the shares in 2016 as in 2018, You then might conclude 2018 had half as much fake-news sharing as 2016. You'd probably be wrong.

More likely the number of shares stayed steady. Half the sites listed in 2016 were gone in 2018. So you'd expect to see about half the shares in 2018 as 2016 if sharing remained the same, (all other factors being equal — which, of course, they're not.) Your conclusion would have made things look good, when, if fact, there as miserable as ever.

Recent studies, brower extensions, and tools that relying on fake-news lists:

Studies

Misinformation in action: fake news exposure is linked to lower trust in media, higher trust in government when your side is in power (BF, FC, PF; 2020-06)
(Uses: "list of domains categorized as fake news by Grinberg and colleagues (2019)" —see )
https://misinforeview.hks.harvard.edu/article/misinformation-in-action-fake-news-exposure-is-linked-to-lower-trust-in-media-higher-trust-in-government-when-your-side-is-in-power/

Why Do People Share Ideologically Extreme, False, and Misleading Content on Social Media? A Self-Report and Trace Data–Based Analysis of Countermedia Content Dissemination on Facebook and Twitter (FNW, OS; 2020-05)
(Uses: "About.com, Aloisius, CBS News, The Daily Dot, Fake News Watch, Fake News Checker, Melissa Zimdars, NPR, Snopes Field Guide, The New Republic, and U.S. News and World Report"; 106 "countermedia" xdomains in Appendix A)
https://academic.oup.com/hcr/article/doi/10.1093/hcr/hqz022/5840447?guestAccessKey=e1548abf-a0ae-469a-98f5-a9a04b0b769e

Political audience diversity and news quality (NG; 2020-02)
https://www.newsguardtech.com/press/newsguard-debuts-news-website-reliability-index-creating-the-first-benchmark-for-measuring-news-reliability-and-media-consumption-habits/
https://cpb-us-w2.wpmucdn.com/express.northeastern.edu/dist/d/53/files/2020/02/CJ_2020_paper_59.pdf

The Quarter Billion Dollar Question: How is Disinformation Gaming Ad Tech? (OS, PF; 2019-09)
(Uses: "Storyful, PolitiFact, Le Monde, Open Brand Safety, OpenSources and NewsTracker")
https://disinformationindex.org/wp-content/uploads/2019/09/GDI_Ad-tech_Report_Screen_AW16.pdf

Iffy Quotient: A Platform Health Metric for Misinformation (MBFC, NG; 2019-07)
https://csmr.umich.edu/wp-content/uploads/2019/07/UMSI-CSMR-Iffy-Quotient-Whitepaper-v2.pdf
https://www.newsguardtech.com/press/university-of-michigans-iffy-quotient-shows-steady-drop-of-questionable-information-on-social-media-partners-with-newsguard-for-better-data
https://csmr.umich.edu/platform-health-metrics/

Detecting Fake News on Social Media (BF, OS; 2019-06)
http://dmml.asu.edu/dfn/
Software and datasets
https://github.com/mdepak/fake-news-detection-resources

Fighting misinformation on social media using crowdsourced judgments of news source quality (BF, OS, PF; 2019-02 )
https://www.pnas.org/content/116/7/2521

Less than you think: Prevalence and predictors of fake news dissemination on Facebook (BF, OS; 2019-01 )
https://advances.sciencemag.org/content/5/1/eaau4586

Fighting misinformation on social media using crowdsourced judgments of news source quality (BF, OS, PF, SN; 2019-01)
https://www.pnas.org/content/early/2019/01/14/1806781116

Fake news on Twitter during the 2016 U.S. presidential election (BF, FC, PF; 2019-01)
(Developed fake-news list used by other reseachers, cited as "Grinberg…") http://science.sciencemag.org/content/363/6425/374
http://science.sciencemag.org/highwire/filestream/721809/field_highwire_adjunct_files/0/aau2706_Grinberg_SM.pdf

Less than you think: Prevalence and predictors of fake news dissemination on Facebook (BF, OS; 2019-01)
http://advances.sciencemag.org/content/5/1/eaau4586

Fake News and Advertising on Social Media: A Study of the Anti-Vaccination Movement (HXY; 2018-11)
http://papers.nber.org/tmp/39859-w25223.pdf

Disinformation, ‘Fake News’ and Influence Campaigns on Twitter (OS, MBFC; 2018-10)
https://knightfoundation.org/reports/disinformation-fake-news-and-influence-campaigns-on-twitter

Iffy Quotient: A Platform Health Metric for Misinformation (OS, MBFC; 2018-10)
https://newsroom.fb.com/news/2018/10/inside-feed-michigan-lemonde/
https://csmr.umich.edu/wp-content/uploads/2018/10/UMSI-CSMR-Iffy-Quotient-Whitepaper-810084.pdf

Trends in the Diffusion of Misinformation on Social Media (PF, OS, BF, FC, SN; 2018-10) https://web.stanford.edu/~gentzkow/research/fake-news-trends.pdf

A Large-scale Study of Social Media Sources in News Articles (IIB, OS, WI; 2018-10)
https://arxiv.org/pdf/1810.13078.pdf

Predicting Factuality of Reporting and Bias of News Media Sources (MBFC; 2018-10) https://admin.govexec.com/media/emnlp-2018-predicting.pdf

NewsWhip: Partisan Publishers & Political Content (PF 2018-09)
http://go.newswhip.com/2018_09-Partisan_LP.html
http://go.newswhip.com/rs/647-QQK-704/images/Hyper-Partisan%20Final.pdf

The spread of low-credibility content by social bots (PF 2018-05)
https://www.poynter.org/news/bots-spread-lot-fakery-during-2016-election-they-can-also-debunk-it https://arxiv.org/abs/1707.07592

CJR/Tow Ctr coverage of studies (that use uotdated fake-news site datasets):
https://www.cjr.org/innovations/study-partisan-trust-media.php
https://www.cjr.org/the_new_gatekeepers/chrome-plugin-fake-news.php
https://www.cjr.org/tow_center_reports/partnership-press-facebook-news-outlets-team-fight-misinformation.php
https://www.cjr.org/analysis/fake-news-facebook-audience-drudge-breitbart-study.php https://www.cjr.org/tow_center_reports/ovadya-credibility-journalism-ocasio.php

Browser extensions

B.S. Detector (OS) http://bsdetector.tech

Fake News Alert (OS) http://nymag.com/intelligencer/2016/11/heres-a-browser-extension-that-will-flag-fake-news-sites.html

Slate (2016, now gone; list: uses their own db: "We rely on crowdsourcing and human moderation by Slate staffers and contributors, who will manage a database of flagged sites and links.")
https://slate.com/technology/2016/12/introducing-this-is-fake-slates-tool-for-stopping-fake-news-on-facebook.html

Fake News Detector (list: ?) https://fakenewsdetector.org/en

Newstrition (list: Freedom Forum Institute and Our.News)
https://www.freedomforuminstitute.org/first-amendment-center/newstrition/

Facebook is patenting a tool that could help automate removal of fake news
https://www.theverge.com/2016/12/7/13868650/facebook-fake-news-patent-tool-machine-learning-content

Check This by MetaCert (MC)
https://medium.com/@Paul__Walsh/mock-this-chrome-extension-takes-aim-at-fake-news-c407320f2835
https://chrome.google.com/webstore/detail/check-this-by-metacert/felmjclcjadopolhjmlbemfekjaojfbn?hl=en
MetaCert Protocol https://metacertprotocol.com/

Trusted News for Google Chrome (MC)
https://trusted-news.com/

NewsGuard (propritary, NG) https://www.newsguardtech.com/

Official Media Bias Fact Check Icon (MBFC)
https://jeffreyatw.com/blog/2016/11/check-news-bias-with-a-simple-browser-icon/
https://mediabiasfactcheck.com/appsextensions/

Read Across The Aisle (rates bias not fake) http://www.readacrosstheaisle.com/

Tools

Junk News Aggregator (list: their own) https://newsaggregator.oii.ox.ac.uk/methodology.php
Junk News Classification
http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/93/2018/02/Polarization-Partisanship-JunkNews-OnlineSupplement.pdf
(Computational Propaganda group (COMPROP) of the Oxford Internet Institute (OII) at the University of Oxford)

Abbreviations

Key to lists:

  • BuzzFeed (BF)
  • Fake News Codex (FNC)
  • FactCheck.org (FC)
  • Hoaxy (HXY)
  • Information is Beautiful (IIB)
  • Is It Fake News? (IIFN)
  • Media Bias / Fact Check (MBFC)
  • MetaCert (MC)
  • NewsGuard (NG)
  • OpenSources.co (OS)
  • PolitiFact (PF)
  • Snopes (SN)
  • Wikipedia (WI)

Details at: Fake and Unreliable News Sites