Example Storm Topologies
Learn to use Storm!
Table of Contents
- Getting started
- Using storm-starter with Leiningen
- Using storm-starter with Maven
- Using storm-starter with IntelliJ IDEA
First, you need
git installed and in your user's
PATH. Also, two of the examples in storm-starter
require Python and Ruby.
Next, make sure you have the storm-starter code available on your machine. Git/GitHub beginners may want to use the following command to download the latest storm-starter code and change to the new directory that contains the downloaded code.
$ git clone git://github.com/nathanmarz/storm-starter.git && cd storm-starter
storm-starter contains a variety of examples of using Storm. If this is your first time working with Storm, check out these topologies first:
- ExclamationTopology: Basic topology written in all Java
- WordCountTopology: Basic topology that makes use of multilang by implementing one bolt in Python
- ReachTopology: Example of complex DRPC on top of Storm
If you want to learn more about how Storm works, please head over to the Storm project page.
Using storm-starter with Leiningen
Running topologies with Leiningen
To run a Java topology
$ lein deps $ lein compile $ java -cp $(lein classpath) storm.starter.ExclamationTopology
To run a Clojure topology:
$ lein deps $ lein compile $ lein run -m storm.starter.clj.word-count
Using storm-starter with Maven
Running topologies with Maven
storm-starter contains m2-pom.xml which can be used with Maven using the
-f option. For example, to
compile and run
WordCountTopology in local mode, use the command:
$ mvn -f m2-pom.xml compile exec:java -Dstorm.topology=storm.starter.WordCountTopology
You can also run clojure topologies with Maven:
$ mvn -f m2-pom.xml compile exec:java -Dstorm.topology=storm.starter.clj.word_count
Packaging storm-starter for use on a Storm cluster
You can package a jar suitable for submitting to a Storm cluster with the command:
$ mvn -f m2-pom.xml package
This will package your code and all the non-Storm dependencies into a single "uberjar" at the path
Running unit tests
Use the following Maven command to run the unit tests that ship with storm-starter. Unfortunately
lein test does not
yet run the included unit tests.
$ mvn -f m2-pom.xml test
Using storm-starter with IntelliJ IDEA
Importing storm-starter as a project in IDEA
The following instructions will import storm-starter as a new project in IntelliJ IDEA.
pom.xml. This is requried so that IDEA (or Eclipse) can properly detect the maven configuration.
- Open File > Import Project... and navigate to the top-level directory of your storm-starter clone (e.g.
- Select Import project from external model, select "Maven", and click Next.
- In the following screen, enable the checkbox Import Maven projects automatically. Leave all other values at their defaults. Click Next.
- Click Next on the following screen about selecting Maven projects to import.
- Select the JDK to be used by IDEA for storm-starter, then click Next.
- At the time of this writing you should use JDK 6.
- It is strongly recommended to use Sun/Oracle JDK 6 rather than OpenJDK 6.
- You may now optionally change the name of the project in IDEA. The default name suggested by IDEA is "storm-starter". Click Finish once you are done.