This is a sample storm topology for stream processing sensor data. In this case, we're applying simple rules to events being generated by a trucking fleet. The goal is to send email alerts to a dispatch center when a threshold of "Un-Safe" events have been generated by a specific truck/driver.
There are three dependencies to building this example:
- Java 1.6
Dependencies 1, 2, and 3 can be downloaded and installed respectively. Once they've been installed and you can do a
mvn -version from a command prompt, I'll cover how to build and install storm-jms, and then this project.
I've forked @ptgoetz's storm-jms project, as one addition was needed to build the project. A pull request has been issued and once it has been added, you can build directly from his repo. For now, we'll go ahead and get it from my repo using the commands below:
git clone https://github.com/pcodding/storm-jms.git cd storm-jms mvn clean install
Now that the storm-jms dependency has been installed, we can build this project. The process is very similar and is as follows:
git clone https://github.com/pcodding/storm-streaming.git cd storm-jms mvn clean install
This project uses the following classes to define the topology
com.hortonworks.streaming.impl.topologies.TruckEventProcessorTopology. It has one spout,
JmsSpout, and one bolt,
TruckEventRuleBolt class receives each
Tuple, and processes it using the
TruckEventRuleEngine class. The
TruckEventRuleEngine class stores each "Un-Safe" event for a driver and sends an email if 5 events have been captured.
src/main/resources/config.properties has the following properties:
firstname.lastname@example.org notification.subject=Unsafe Driving Alerts jms.host=sandbox jms.port=61616 jms.notificationQueue=sensor_data jms.notificationTopic=sensor_events
notification.* are related to the emails sent after the event threshold has been reached, and the
jms.* configuration properties control the JMS connection and system details. Make sure you edit the
notification.email before running the project.
Running the Project
To run the project, just use the
run.sh shell script to compile and execute the project on a stand-alone machine.
What's a Storm topology without events? Not too exciting. I've created a data simulator that helps with simulating unsafe driving events among other things. You can find it here, and this is the process to get started:
git clone https://github.com/pcodding/stream-simulator.git
src/main/resources/config.properties to point to your ActiveMQ installation, then build and run the project:
mvn clean package ./run.sh 6 -1 com.hortonworks.streaming.impl.domain.transport.Truck com.hortonworks.streaming.impl.collectors.JmsEventCollector