About

A big data group project that includes data manipulation, prediction parameters configuration with ALS and, lastly, communities detection with Newman-Girvan algorithm. The data used is the 20M MovieLens Dataset.

Authors

1. Alexandros Rantos
2. Duaa Alqattan
3. Alexander Merschel

Technologies

This project was developed in Databricks 6.3 & Apache Spark 2.4.4.

Furthermore, dataframes and pandas framework are also used.

Dependencies

pip install pyspark
pip install graphframes
pip install dataframe
pip install pandas