This package is part of my Insight Project nexTop.
The library runs a Spark Streaming consumer that receives messages posted by the Kafka S3 Producer and sends them into Elasticsearch using the native client library elasticsearch-hadoop.
To build the jar:
sbt assembly
Example usage in standalone:
$SPARK_HOME/bin/spark-submit --class insightproject.spark.newsstreaming.NewsStreaming --master \
spark://`hostname`:7077 --jars newsStreaming-assembly-1.0.jar newsStreaming-assembly-1.0.jar \
my_topic my_group_id elastic_destination_index/elastic_doc_type