Deploy a Spark/Yarn/Hadoop Cluster with Gaia DR2 on Openstack

[stvoutsin]

• Create a deployment on a large enough Openstack cluster (dev)
• Fetch Gaia Source DR2 csv files from http://cdn.gea.esac.esa.int/Gaia/gdr2/gaia_source/csv/
• Transform Gaia Source DR2 into Parquet

[Zarquan]

Need detailed notes on how this was done so we can replicate it.

[Zarquan]

Split this issue into two parts, the first part is about the deployment of Spark&Hadoop onto OpenStack virtual machines, and then the transform csv into Parquet can be handled in a separate issue (#31 ).

[Zarquan]

Closing this issue - we have a manual deployment running on the gaia_dev project.