Word Count in Spark - Write output to file
This repo contains a word count program that writes output to file.
Clone this repo
Uncomment line 14 when running on local. This line is commented so that we can use Docker master.
//.master("local") //uncomment this line when running on local
Build the project by running -
gradle clean build
spark-submit --master local --verbose --class com.pavanpkulkarni.dockerwordcount.DockerWordCount build/libs/Docker_WordCount_Spark-1.0.jar <input_filename> <output_directory>
spark-submit --master local --verbose --class com.pavanpkulkarni.dockerwordcount.DockerWordCount build/libs/Docker_WordCount_Spark-1.0.jar "data.txt" "output"
Output will be available under