Repository navigation
Home
In this project we examined how recommender systems work (better or worse) if we take advantage of the review texts along with the review ratings. We aimed to combine latent ratings with latent review topics and analyze the results. Our assumption was that combining the review text with ratings would help the recommender system make better predictions. Hence, we compared different models on Amazon Apps dataset and calculated RMSE.
In today’s world of ubiquitous digital content, the number of choices we have are overwhelming. In an effort to understand how recommender systems can make better predictions, we decided to explore the impact of review text on suggesting relevant items to users and helping them discover the content they are looking for. We hypothesized that using review text would produce better recommendations than traditional ratings-only based approaches to recommendation systems.
As visible from the table below, using the review text in combination with the numerical ratings hailed better results in terms of both RMSE and MAE. The HFT model performed the best with an RMSE of 1.1 on the whole 750K reviews, and an RMSE of 1.02 on 20K reviews. We did not run the Collaborative Filtering model or Latent Dirichlet Allocation model on the 750K dataset. Additionally, it is interesting to note that most of our baseline methods (using only ratings) performed either better or roughly the same on the larger dataset, whereas the HFT model performed slightly worse.
| Approach 0 (Global Mean) | Approach 1 (Biases) | Approach 2 (CF) | Approach 3 (SVD) | Approach 4 (LDA) | Approach 5 (HFT) | |
|---|---|---|---|---|---|---|
| RMSE | 1.32 | 1.23 | Mexico | 1.18 | Francisco Chang | 1.1 |
| MAE | 1.06 | 0.93 | Mexico | 0.9127 | Francisco Chang | 0.85 |
| Approach 0 (Global Mean) | Approach 1 (Biases) | Approach 2 (CF) | Approach 3 (SVD) | Approach 4 (LDA) | Approach 5 (HFT) | |
|---|---|---|---|---|---|---|
| RMSE | 1.26 | 1.37 | 1.33 | 1.18 | 1.4 | 1.02 |
| MAE | 0.98 | 1.00 | 0.93 | 0.91 | 1.03 | 0.76 |
Using review topics in conjunction with ratings gives better recommendations than traditional methods
It is difficult to make accurate predictions by relying on review text and biases alone (LDA)
HFT is dataset dependent – some categories of items have reviews that better express the subjective tastes of the user/properties of the item
Beyond a certain value, increasing the number of topics gives negligible improvements in RMSE
App reviews are short and to-the-point, and mostly positive
- Creating a test set such that all items & users are in the train set too.
- Creating an automated Amazon ASIN lookup tool which is usually a paid service
- LDA Topics are not well defined for the dataset we chose.
- No ground truth for the predictions by using review texts only.