Project proposal

https://mailmissouri-my.sharepoint.com/:p:/g/personal/bacxf_mail_umkc_edu/EUIuKD8Urz5DrTxDaDPEIlsBPoDDaEFDtganeuQ97UFstA?e=GDIlAB

we are working on

1. hive mapreduce (module 1)
2. hive with tez (module 1)
3. spark SQL (module 2)
4. spark dataframe(module 2)

• We want to check the run time of queries like ’like’ , ‘group by’ ,’order by’ , ‘count’ , ‘pattern-match’ , etc and help the users tell which one is best for which query and save their time from selecting which tool to be used.
• I’m using the flight data set called ‘data expo 09’
• I’m setting up hive, spark environment in ITversity labs ( CREDENTIALS : Username : apandit7 Password : Ik0i0a6ti4aenguezei2Chie3Eupie0d ) It has around 400GB of ram and it just costed me around 29$.
• I have run the like query in Hive , Hive with Tez, SparkDataFrame , SparkSQL.