Skip to content

alagrede/Kafka-Sparkstreaming-example

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 

Repository files navigation

Kafka example

kafka-producer: Simple Kafka producer

sparktreaming-kafka-consumer: SparkStreaming job consumer with kafka

Kafka installation

https://kafka.apache.org/quickstart

SparkStreaming installation

https://spark.apache.org/downloads.html

Download winutils.exe from the repository to some local folder, e.g. C:\hadoop\bin. https://github.com/steveloughran/winutils

Set HADOOP_HOME to C:\hadoop. Open command prompt with admin rights. Run C:\hadoop\bin\winutils.exe chmod 777 /tmp Run C:\hadoop\bin\winutils.exe chmod 777 /Temp Download and install Spark on Windows (add the bin folder to the PATH)

About

Kafka example producer with a SparkStreaming consumer job

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages