Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.


This is a somewhat cleaned up version of the code I presented at SpringOne2014 to show how Spark can be used in Spring apps. It's based on an app I'm writing to do real time and processing of a data stream coming from a high powered model rocket. The goal of this sample is Spring and Spark, so most of the rocket code was eliminated.

What is left is a somewhat generic Spring Boot application. The app will run with a webui for local experimentation, or as a standalone app for running on a cluster. The behavior is controlled by the property. For running in eclipse/STS I typically use the webui, and the local data files in the project. (The default is to use HDFS.)

In the run profile pass these parameters

For a simple run profile that just processes one of the data files pass (or 0)

Deploy Spark Cluster

I have not tested the app deployed to a cluster yet. The sparkjar task that created the "UberJar", is still including the spark and hadoop clients. After I get that fixed, I'll test it and deploy a version that's been tested on a cluster.

You can’t perform that action at this time.