Kafka demos at Scalathon
Scala Shell
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
bin
config
lib
project
scala-tutorial-scripts
src/main
.gitignore
README.md
sbt
scalathon.iml initial commit Jul 17, 2011

README.md

Kafka demo at Scalathon

This repository has the Kafka demos used at Scalathon.

How to build ?

The demo uses SBT 0.7.5

./sbt update package

How to run ?

You need to start a zookeeper and kafka server. See the kafka quickstart page for instructions on how to do that

To run the Kafka demo, see the scripts provided in the bin directory

Kafka producer examples

  • Run the basic producer examples

    bin/basic-producer-example.sh

    Run the kafka consumer shell. See quickstart for more details

    kafka/bin/kafka-consumer-shell.sh --topic test-topic --props config/consumer.properties

  • Run the producer encoder example

    `bin/producer-encoder-example.sh

    Run the kafka consumer shell to consume that data

    kafka/bin/kafka-consumer-shell.sh --topic member-records --props config/consumer.properties

  • Run the producer partitioner example

    bin/producer-partitioner-example.sh

    Spin up another kafka consumer in the same group and topic to see automatic load balancing amongst the two consumers

    kafka/bin/kafka-consumer-shell.sh --topic member-records --props config/consumer.properties

    Run the producer partitioner again and observe, all the "US" location records go to one consumer, while all "EUR" records go to the other consumer

Kafka consumer examples

  • Run the consumer decoder examples to consumer all data from the "member-records" topic

    bin/consumer-decoder-example.sh

Want to contribute to Kafka ?

For project ideas to contribute to Kafka, see our projects page

Send us a pull request here