docker-spark

Minimal setup to run Spark on Docker

Requirements

• Docker

Getting started

• Enable your Docker deamon
• Run make up: it will build all the necessary Docker images, and lunch a mini spark cluster with 2 nodes. Another container will be running jupyterlab

Connect to jupyterlab to interact with Spark or to spark-master to check the Spark cluster status.

Note

• The first build can take up to 10 minutes, because Python is downloaded and compiled (step to imrpove)