Skip to content
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Vagrant for Hadoop, Spark and Hive


Vagrant project to spin up a single virtual machine running:

  • Hadoop 2.7.2
  • Hive 1.2.1
  • Spark 1.6.0

The virtual machine will be running the following services:

  • HDFS NameNode + NameNode
  • YARN ResourceManager + JobHistoryServer + ProxyServer
  • Hive metastore and server2
  • Spark history server

Getting Started

  1. Download and install VirtualBox
  2. Download and install Vagrant.
  3. Run vagrant box add centos65
  4. Go to releases and download and extract the latest source of this project.
  5. In your terminal change your directory into the project directory (i.e. cd vagrant-hadoop-spark-hive-<version>).
  6. Run vagrant up to create the VM.
  7. Execute vagrant ssh to login to the VM.

Web user interfaces

Here are some useful links to navigate to various UI's:

Validating your virtual machine setup

To test out the virtual machine setup, and for examples of how to run MapReduce, Hive and Spark, head on over to

Starting services in the event of a system restart

Currently if you restart your VM then the Hadoop/Spark/Hive services won't be up (this is something I'll address soon). In the interim you can run the following commands to bring them up:

$ vagrant ssh
$ sudo -s
$ /vagrant/scripts/
$ nohup hive --service metastore < /dev/null > /usr/local/hive/logs/hive_metastore_`date +"%Y%m%d%H%M%S"`.log 2>&1 </dev/null &
$ nohup hive --service hiveserver2 < /dev/null > /usr/local/hive/logs/hive_server2_`date +"%Y%m%d%H%M%S"`.log 2>&1 </dev/null &
$ /usr/local/spark/sbin/

More advanced setup

If you'd like to learn more about working and optimizing Vagrant then take a look at

For developers

The file contains some tips for developers.


This project is based on the great work carried out at (


Vagrant project to spin up a single virtual machine running current versions of Hadoop, Hive and Spark




You can’t perform that action at this time.