RADio

• 1 Computing RADio with rank-aware JS-divergence
• 2 from scratch
• 3 Additional Material
  • 3.1 Jensen Shannon as an f-Divergence
  • 3.2 RADio with KL divergence
  • 3.3 Jensen-Shannon divergence Kullback-Leibler divergence with and without rank-awareness
  • 3.4 Jensen-Shannon divergence for all recommender strategies without cutoff

This repository contains the supporting material for the paper 📻 RADio – Rank-Aware Divergence metrIcs to measure nOrmative diversity in news recommendations.

1 Computing RADio with rank-aware JS-divergence

1. download articles_large.pickle: https://drive.google.com/file/d/1XgXetSxweN9rLo6vCMMR5ur5lvYWJYCY/view?usp=sharing
2. download recommendations_large_all_recs_no_cap.pickle: https://drive.google.com/file/d/17X9fkhBxzNhHhWgb_XHQJ1WVexhnsZnX/view?usp=sharing
3. Put articles_large.pickle and recommendations_large_all_recs_no_cap.pickle in the data folder
4. download behaviors_large.tsv: https://drive.google.com/file/d/13ENNPX7f7Qb2vLtdp94Zo5lBflch_hiJ/view?usp=sharing
5. Put behaviors_large.tsv in the data/recommendations folder

git clone https://github.com/svrijenhoek/RADio.git
pip install -r requirements.txt
python metrics_calculation.py

2 from scratch

MacOS

brew tap elastic/tap
brew install elastic/tap/elasticsearch-full

python -m spacy download en_core_web_sm

3 Additional Material

3.1 Jensen Shannon as an f-Divergence

3.2 RADio with KL divergence

RADio framework with DART metrics based on KL divergence with recommendation algorithms on the MIND dataset. From left to right: Calibration (topic), Calibration (complexity), Fragmentation, Activation, Representation and Alternative Voices. These metrics are executed on a random sample of 35.000 users, with rank-awareness, and without cutoff point.

rec_type	calibration_topic	calibration_complexity	fragmentation	affect	representation	alternative_voices
lstur	2.6038	1.1432	7.7201	0.1481	0.1078	0.0142
naml	2.5333	1.1287	7.3926	0.1531	0.1047	0.0127
npa	2.5945	1.1390	7.6202	0.1521	0.1237	0.0134
nrms	2.5013	1.1204	7.4519	0.1442	0.1114	0.0113
pop	2.9384	1.1082	7.6377	0.1605	0.1028	0.0102
random	3.6038	1.5985	8.6295	0.8079	1.1248	0.0420

3.3 Jensen-Shannon divergence Kullback-Leibler divergence with and without rank-awareness

3.4 Jensen-Shannon divergence for all recommender strategies without cutoff