[💡SUG] Add better simple non-neural baselines

[deklanw]

The recent "A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research" https://arxiv.org/abs/1911.07698 found that relatively simple non-neural baselines which were properly tuned outperformed basically all the neural approaches they tested.

Looking at the results in the Appendix the best performing of those baselines seem to be:

• EASE paper
• SLIM paper code
• RP3Beta paper

All the baselines are also implemented in the repo for the aforementioned paper here https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluation

I've already implemented EASE for RecBole here #609 (it is indeed easy). I may try doing the others as well.

And, btw, the best performing neural method in their tests was Mult-VAE, which I see is underway here #603

I assume that looking for papers which cite the papers of these methods is a good start for finding promising algorithms. Some examples,

• Boosting Item-based Collaborative Filtering via Nearly Uncoupled Random Walks (2020) paper code
• Personalized diffusions for top-n recommendation (2019) paper code video
• Block-Aware Item Similarity Models for Top-N Recommendation (2020) paper

If I find more I'll probably add them here

[batmanfly]

@deklanw Yes, VAE based methods are under development, and they would be ready online around mid January (code review and performance evaluation would take some time).

Based on my experiences, VAE based or non-sampling methods have a natural advantage in achieving very good recommendation performance. A possible reason is that it can refer to the whole item set when training an user-item interaction record. It is particularly suitable to the case that all the test items are seen in training set.

Yes, if you want more methods to be included, please update them at this issue. We expect to get more hands to develop new algorithms after mid January (after the final-exam period at our college). It would be also great if you could make code contribution on these algorithms.

[deklanw]

@batmanfly Good news

Although, I should note that most of those I listed aren't autoencoders. The two papers I referenced by Nikolakopoulos are random walk based. RP3Beta is also random walk based, and sometimes outperforms EASE in the "Troubling Analysis" paper above.

And, in my tests so far with RecBole the best performing general recommender is LightGCN which is of course not an autoencoder nor random-walk based. Microsoft Recommenders has a tiny 100k/1m MovieLens benchmark going and LightGCN is also the best of those: https://github.com/microsoft/recommenders/blob/14225cea3339ba779943097c4890405814aaffa3/examples/06_benchmarks/movielens.ipynb (table at bottom).

Do you know of an autoencoder which gets better results than LightGCN (or, anything really)?

I'll try implementing some more