Skip to content
Charmander Scheduler Lab - Mesos, Docker, InfluxDB, Spark
Shell Go Ruby
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
analytics
bin cleaning up the move to pcp running in a container Mar 16, 2016
cadvisor Undo Upgraded versions of underlying software Oct 7, 2015
docs cleaning up the move to pcp running in a container Mar 16, 2016
experiments Breaking out the experiments in to their own repositories Jul 16, 2015
lib New - added optional traffic generator node type Aug 5, 2015
vector Merge branch 'master' of https://github.com/att-innovate/charmander Mar 16, 2016
.gitignore Update .gitignore Oct 7, 2015
LICENSE
README.md
Vagrantfile upgrade mesos to 0.28 May 30, 2016
cluster.yml
node_traffic.txt cleaning up the move to pcp running in a container Mar 16, 2016

README.md

Charmander Scheduler Lab

Charmander is a lab environment for measuring and analyzing resource-scheduling algorithms.

The project got started in Summer 2014 by Theodora Chu as an internship project. It was motivated by a paper from Stanford University: "Quasar: Resource-Efficient and QoS-Aware Cluster Management".

Charmander at its core provides an easy to use environment to a) schedule and deploy compute-tasks in a multi-node setup and b) measure the corresponding cpu, memory, and network-loads.

image

The collected measurements can afterwards be analyzed using the included Spark analytics workbench and subsequently those results can be fed back in to the scheduler.

Obviously this lab-setup can be used for other use-cases like testing and analyzing machine-learning based anomaly-detection, noisy-neighbor detection, or profiling algorithms, or simply serving as the load-pattern verification authority in a continuous integration environment.

Setup and Run Charmander

All that is required to run a simple lab setup and an experiment is Vagrant, VirtualBox, curl, a fast internet connection, and a bit of spare time.

All the steps are automated and are part of simple scripts that come with the Charmander project. All the additional software needed during the setup gets installed and built inside the VMs, nothing additionally gets installed on the host itself.

Btw, if at any time during installation you run in to an error, please follow the steps outlined in Tips and Tricks

  1. Setup Nodes

    • Configure and build the different nodes with Vagrant and VirtualBox
    • Reload and reset environment
  2. Build and deploy Scheduler

    • Clone the Charmander Scheduler projects
    • Compile it inside the master node
    • Deploy and run it
  3. Build and deploy Analytics Stack

    • Build all the Docker images for the full analytics stack on the analytics node
  4. Run some Experiments

  5. Scheduler API

    • List of all the different Scheduler REST APIs
  6. Script Reference

    • List of all the include scripts/tools
  7. Tips and Tricks

    • Some tips and tricks for common issues

The different Charmander projects on GitHub

Additional thanks goes to

some additional open source projects and a blog post that have inspired us:

You can’t perform that action at this time.