Skip to content
Spark app that demonstrates reading and writing data to from MongoDB and BSON files
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
src/main/java/com/mongodb/spark/demo updates for mvn artifacts and some logging Sep 18, 2014
.gitignore first pass at Spark Recommender that reads BSON and writes back to Mo… Jun 1, 2014
LICENSE minor edits Jun 21, 2014
pom.xml updates for mvn artifacts and some logging Sep 18, 2014

MongoDB-Spark Demo


To build the MongoDB-Spark demo application, you'll need to have the following:


Note: you must build the MongoDB-Hadoop connector from source for your version of Hadoop and install the core JAR file to your local Maven repo. For example:

$ git clone
$ cd mongo-hadoop
$ ./gradlew jar -Phadoop_version='2.4'
$ mvn install:install-file \
    -Dfile=core/build/lib/mongo-hadoop-core-1.2.1-SNAPSHOT-hadoop_2.4.jar \
    -DgroupId=com.mongodb \
    -DartifactId=hadoop \
    -Dversion=1.2.1-SNAPSHOT \


Refer to the Spark overview to get started.


To the build the MongoDB-Hadoop demo applications use Maven:

$ mvn package

This will build the demo application and place all of the dependencies in target/lib. If instead you want to build a single jar with all of the dependencies, execute the assembly:single Maven goal:

$ mvn compile assembly:single


$ cd your-spark-directory
$ SPARK_JAR=assembly/target/scala-2.10/spark-assembly-1.0.0-hadoop2.4.0.jar \
  HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop \
  bin/spark-submit --master local --class com.mongodb.spark.demo.Recommender /path/to/demo-1.0-SNAPSHOT.jar \
  --jars /path/to/mongo-java-driver-2.12.2.jar,/path/to/hadoop-1.2.1-SNAPSHOT.jar \
  --executor-memory 4G /movielens/ratings.bson /movielens/users.bson \
  /movielens/movies.bson movielens.predictions


None at this time.

You can’t perform that action at this time.