Navigation Menu

Skip to content

PocketCluster/pocketcluster

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

74 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

icon

PocketCluster v0.1.3

A one-step utility to setup a multi-nodes BigData cluster on OSX.

PocketCluster will soon be available after fixing few bugs. Please contact @stkim1 or join the slack channel for further detail.

Current Packages Supported

Package Version Release Note
Apache Spark 1.5.2 RN 1.5.2
Apache Hadoop 2.4.0 RN 2.4.1

Supported Platform

OSX El Capitan

This installer has only been tested on El Capitan. May work fine on Yosemite 10.10.

System Setup

In order for Spark/Hadoop to work, you need to open SSH server just for yourself.

  1. Open "System Preference"
  2. Open "Sharing"
  3. Check "Remote Login" for "Administators" only.

Common Pre-requisites

Java

  • Java 8

Homebrew

For Vagrant/VirtualBox Cluster

  • Install following two additional requirements.
  • Mac with at least 8GB Memory and 10 GB of free disk space is strongly recommended.

VirtualBox

Vagrant

Video Instruction

Build Hadoop Cluster on Vagrant

For Raspberry PI 2 Cluster

  • All Raspberry PI 2 and a Mac should be behind the same router.
  • Ethernet Connection required.
  • Must use the image provided below.
  • YOU DO NOT NEED TO CONFIGURE A SINGLE THING FOR ALL RASPBERRY PIs!!!

Rasperry PI 2 Ubuntu 14.04 Image

  • 2015-11-13-ubuntu-trusty.img.zip (232 MB)
  • Checksum: 687a2e9b6c04f7a35449cc95405946e2

(Optional) ApplePi OSX Image Baker

Video Instruction

Build Hadoop Cluster on Raspberry PI

Uninstall (Raspberry PI)

  1. Download uninstall script by right click and Save link as....
  2. Open console or iTerm, go to where the file is, and run following command.

chmod +x ./rpi-uninstall && ./rpi-uninstall

Updates & Questions

You can follow how and what progresses are made at https://pocketcluster.wordpress.com.
If you have a question, tweet me @stkim1.

PocketCluster is Copyright (C) 2015 Sung-Taek, Kim, All Rights Reserved.