Skip to content

julien-diener/sparkwebapp

master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

A very simple spring web app that use apache spark

The goal is to take the spark/hdfs csvconverter and make it run from a simple spring servlet.

The main trouble come from conflict between the spark and hadoop dependencies, and maybe the spring ones. In the previous stand alone app csvconverter, the dependencies were only used for compilation, but at run time another, global, jar is used: the $SPARK_HOME/lib/spark-assembly-1.1.1-hadoop2.4.0.jar provided by the spark installation.

The solution used here is to replace the maven dependencies included in the war by this spark-assembly jar. See this SO answer for more details.

Author Julien Diener

About

A very simple spring web app that use apache spark

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages