end-To-end Adaptive Local Learning (TALL)

MoE with MultVAE as Expert

---

Dataset

Dataset	# Users	# Items	# Ratings	Sparsity
ML1M	6,040	3,952	1,000,209	95.81%
Amazon CDs & Vinyl	24,179	27,602	4,543,369	99.93%
Yelp	25,677	25,815	731,671	99.89%

The table contains the information of original non-preprocessed datasets.

I uploaded one public benchmark datasets code sample here: MovieLens 1M (ML1M). You can find other datasets and run them in this code. We convert all explicit ratings to binary values, whether the ratings are observed or missing, some example datasets are listed in the table above.

You can get the original datasets from the following links:

Movielens: https://grouplens.org/datasets/movielens/

Amazon Review Data: https://nijianmo.github.io/amazon/

Yelp 2015: https://github.com/hexiangnan/sigir16-eals/tree/master/data

---

Basic Usage

• Change the experimental settings in main_config.cfg and the model hyperparameters in model_config.
  
• Run main.py to train and test models.
  
• Command line arguments are also acceptable with the same naming in configuration files. (Both main/model config)

For example: python main.py --model_name MultVAE --lr 0.001

Running LOCA

Before running LOCA, you need (1) user embeddings to find local communities and (2) the global model to cover users who are not considered by local models.

1. Run single MultVAE to get user embedding vectors and the global model:

python main.py --model_name MultVAE

2. Train LOCA with the specific backbone model:

python main.py --model_name LOCA_VAE

Running TALL (MoE)

python main.py --model_name MOE

---

Requirements

• Python 3.7 or higher
• Torch 1.5 or higher

Reproducibility

All models are implemented in PyTorch and optimized by the Adam algorithm. For the baseline MultVAE and the MultVAE component in other models, we set one hidden layer of size 100. In addition, for the TALL, we set the gap window $T$ to 40 with $\alpha=0.08$ for the ML1M; the gap window $T$ to 15 with $\alpha=0.6$ for Yelp; and the window gap $T$ to 25 with $\alpha=0.3$ for CDs&Vinyl. These hyperparameters were tuned by the validation sets. And we set the number of experts in TALL as 100 for all datasets. Likewise, for LOCA and EnLFT, we maintain the number of anchor models at 100 to ensure a fair comparison.

Citation

cited papaer:

@inproceedings{pan2024countering,
  title={Countering Mainstream Bias via End-to-End Adaptive Local Learning},
  author={Pan, Jinhao and Zhu, Ziwei and Wang, Jianling and Lin, Allen and Caverlee, James},
  booktitle={European Conference on Information Retrieval},
  pages={75--89},
  year={2024},
  organization={Springer}
}