No description, website, or topics provided.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
project minor tweaks Oct 19, 2014
src minor tweaks Oct 19, 2014 Update Oct 19, 2014
build.sbt remove unused dependencies + stop context after test Oct 19, 2014

Spark Testing Example Application

Source code for blog posts:


By default tests are running with Spark in local mode

sbt test

If you want to run tests in standalone Spark cluster you need to provide spark.master url

sbt -Dspark.master=spark://spark-host:7777 test-assembled


In the root directory run:

sbt assembly

The application fat jars will be placed in:

  • target/scala-2.10/spark-testing-example-app.jar


First you need to run assembly in sbt and then run java cmd

java -Dspark.master=spark://spark-host:7777 spark-testing-example-app.jar