MongoDB to Spark connector example
This project demonstrate how to use the MongoDB to Spark connector.
The queries are adapted from the aggregation pipeline example from the MongoDB documentation.
How to run:
- Install docker and docker-compose
- Install maven
Run MongoDB and import data
From the project root:
docker-compose -f docker/docker-compose.yml up -d docker exec -it mongo_container sh /scripts/import-data.sh
Verify the data have been loaded in MongoDB by connecting to the container and run a count:
docker exec mongo_container mongo --eval "db.zips.count()"
MongoDB shell version: 3.2.11 connecting to: test 29353
Import the project
- Import the maven project in your favorite IDE
- Run the MongoSparkMain class