A spark cluster based on docker-compose.
-
Updated
Mar 23, 2018 - Shell
A spark cluster based on docker-compose.
A spark cluster containing multiple spark masters based on docker-compose.
This is a self-documentation of learning distributed data storage, parallel processing, and Linux OS using Apache Hadoop, Apache Spark and Raspbian OS. In this project, 3-node cluster will be setup using Raspberry Pi 4, install HDFS and run Spark processing jobs via YARN.
Add a description, image, and links to the spark-cluster topic page so that developers can more easily learn about it.
To associate your repository with the spark-cluster topic, visit your repo's landing page and select "manage topics."