A very simple spring web app that use apache spark
Java
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
src/main
.gitignore
README.md
pom.xml

README.md

A very simple spring web app that use apache spark

The goal is to take the spark/hdfs csvconverter and make it run from a simple spring servlet.

The main trouble come from conflict between the spark and hadoop dependencies, and maybe the spring ones. In the previous stand alone app csvconverter, the dependencies were only used for compilation, but at run time another, global, jar is used: the $SPARK_HOME/lib/spark-assembly-1.1.1-hadoop2.4.0.jar provided by the spark installation.

The solution used here is to replace the maven dependencies included in the war by this spark-assembly jar. See this SO answer for more details.

Author Julien Diener