Project based on COMP530 Big Data Group Project module at University of Liverpool
To demonstrate the validation of the user defined hypotheses using Machine learning algorithm about climate change in the UK :
- Winter will be wetter : To test if the total number of days during winter having heavy rainfall increases
- Summer will be warmer : To test if the total number of days during summer have temperature more than 18 degree increases
- Summer will be drier : To test if the total number of days during summer having light rainfall decreases
- Annual Sunshine Duration will increase : To test if the total number of hours of sunshine annually will increase
- Snowfall in Winter will decrease : To test if the total amount of snowfall during winter will decrease
-
Follow the User Manual.pdf to run the notebook
-
If you do not have access to Hartree SAFE account, then you need to download the data sets from Met Office Website https://www.metoffice.gov.uk/climate/uk/data/ukcp09 and upload it to your HDFS, install spark and pyspark dependencies and have high computing resources such as RAM, multi-core CPU's and sufficient storage (preferably HPC)