Puppet module to install and manage Storm.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
manifests
templates
README.md

README.md

puppet-storm

A puppet module for installing and running Storm.

Note: This puppet module is meant to work with .debs built by storm-deb-packaging.

Puppet Module Installation

Clone (or copy) this repository into your puppet modules/storm directory:

git clone git://github.com/wikimedia/puppet-storm.git

Or you could also use a git submodule:

git submodule add git://github.com/wikimedia/puppet-storm.git modules/storm
git commit -m 'Adding modules/storm as a git submodule.'

Usage

Storm installation and config

All Storm nodes (nimbus, supervisor, ui, etc.) should include the storm class. This ensures that all nodes share the same storm.yaml config file.

class analytics::storm {
  class { "storm":
      nimbus_host     => "nimbus.mydomain.org"
      zookeeper_hosts => ["zk1.mydomain.org", "zk2.mydomain.org"],
      worker_count    => $processorcount - 2, # use all but 2 processors on each worker machine
      ui_port         => 6999,
  }
}

Storm Nimbus Master

# inherit from analytics::storm to get Storm installtion and configs
class analytics::storm::master inherits analytics::storm {
  include storm::nimbus
}

Storm Supervisor Workers

# inherit from analytics::storm to get Storm installtion and configs
class analytics::storm::worker inherits analytics::storm {
  include storm::supervisor
}

Storm Web UI

# inherit from analytics::storm to get Storm installtion and configs
class analytics::storm::ui inherits analtyics::storm {
  include storm::ui
}

Requirements

This module requires that a storm package is available to your package management system. Storm requires ZeroMQ, so it must be available for installation as well.

I built Storm and dependent .debs using storm-deb-packaging, which is forked and modified from the original.

Wishlist

  • Puppetize Storm DRPC.
  • Parameterize JMX Remote Ports.