It's unusual to have opportunity working on server farm, but it's not that difficult to build a cluster by yourself, with some commodity machines, like your old PC, not-use-anymore laptop, or even Raspberry Pi. It's not a bad idea to build such a cluster and try Apache Spark together with Hadoop on it.
In this document, I'm going to introduce the process to set up a cluster by yourself, including the hardware needed, network configuration, software installation, as well as some issues I encountered. It may be helpful for you too.
- Hardware Requirement
- Network Basics
- Set Up Hadoop Distributed File System
- Set Up Apache Spark
- Running Example
- Common Issues
- References
