Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
project
src/main/scala/poc/svend
.gitignore
README.md
build.sbt

README.md

Kakfa Streams one-to-many join

WIP...

Generate some test data

Make sure to start a Kafka broker (I just used the Confluent platform on my laptop).

Create the necessary topics:

 kafka-topics --create --topic car-events \
                     --zookeeper localhost:2181 \
                     --partitions 1 \
                     --replication-factor 1
 
 kafka-topics --create --topic zone-events \
                     --zookeeper localhost:2181 \
                     --partitions 1 \
                     --replication-factor 1
                      
kafka-topics --create --topic car-move-events-partitioned-by-zone \
                     --zookeeper localhost:2181 \
                     --partitions 1 \
                     --replication-factor 1

Then simply this to start generating fake traffic:

sbt runMain poc.svend.FakeData

If you want, you can check that data is correctly arriving:

 kafka-console-consumer \
    --bootstrap-server localhost:9092 \
    --from-beginning  \
    --topic car-events
kafka-console-consumer \
    --bootstrap-server localhost:9092 \
    --from-beginning  \
    --topic zone-events

Start the demo

Just run:

sbt runMai poc.svend.DemoApp

Between executions, make sure to reset the offsets to re-read everything at every run.

kafka-streams-application-reset \
    --application-id one-to-many-join-demo \
    --bootstrap-servers localhost:9092 \
    --to-earliest \
    --input-topics car-events,zone-events \
    --intermediate-topics car-move-events-partitioned-by-zone
You can’t perform that action at this time.