Skip to content
Branch: master
Find file History

Latest commit

gAmUssA and tlberglund Tim's AWS Reinvent demo (#110)
* Initial setup

- added containers with mysql and cp
- populated movies

* added «legacy» app exaple

* fixed some gradle compilation issues

* adding replicator to ccloud for ratings and movies

* added debezium to the game

* RestController first step

* adding troubleshooting commands

* Got RestController working

* Changed topic name in replicator config

* cutomizing debezium, adding correct config for replicator

* Got replicator working

* Added ksqlDB to things

* Additional scripts

Co-authored-by: Tim Berglund <tlberglund@gmail.com>
Latest commit 080f611 Apr 1, 2020

Files

Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
core/src/main Tim's AWS Reinvent demo (#110) Apr 1, 2020
data [Streaming Movie Ratings] Added Interactive Queries demo and Kotlin v… May 29, 2019
gradle/wrapper Tim's AWS Reinvent demo (#110) Apr 1, 2020
legacy-movies-app Tim's AWS Reinvent demo (#110) Apr 1, 2020
legacy-movies-docker Tim's AWS Reinvent demo (#110) Apr 1, 2020
loader/src/main/groovy Tim's AWS Reinvent demo (#110) Apr 1, 2020
spring-streams/src Kubernetes improvements (#77) Mar 19, 2020
spring-web-loader/src/main Tim's AWS Reinvent demo (#110) Apr 1, 2020
streams-kotlin/src/main/kotlin Kubernetes improvements (#77) Mar 19, 2020
streams-kubernetes Kubernetes improvements (#77) Mar 19, 2020
streams/src Kubernetes improvements (#77) Mar 19, 2020
workshop Change all CONTROL_CENTER_CONNECT configs to fully formed URLs (c.f. … Jul 8, 2019
.gitignore Streaming Movie Ratings for Kubernetes (#63) May 6, 2019
README.adoc Added link to playbook Nov 21, 2018
build.gradle Tim's AWS Reinvent demo (#110) Apr 1, 2020
cli-install.properties Made avro utils more Docker-compatible Apr 16, 2019
gradle.properties Switch to CLI-based properties file Nov 20, 2019
gradlew
gradlew.bat Streaming Movie Ratings for Kubernetes (#63) May 6, 2019
settings.gradle Tim's AWS Reinvent demo (#110) Apr 1, 2020

README.adoc

Kafka Streams and KSQL Demo

Note
NEW: Full playbook is here

This repo is a storehouse of bits of code useful for demonstrating Kafka Streams and Confluent KSQL. By way of expectation management, it’s not a coherent demo project as such, but it does have useful things in it that you might want to know how to use.

Building

The main Gradle build file names all the dependencies needed to build the project. The build file doesn’t contain any tasks to run the demo, but is useful as an illustration

Running gradlew idea will generate IntelliJ IDEA project files. It is from there that I typically run StreamsDemo.java, which has a main() method containing a simple Kafka Streams application.

Test Data

The data directory contains two strangely-formatted data files, movies.dat and ratings.dat. These are parsed by io.confluent.demo.Parser into Avro objects called io.confluent.demo.Movie and io.confluent.demo.Rating. This all happens for free in StreamsDemo.java.

That directory also contains JSON versions of the data called movies-json.js and ratings-json.js. These are handy for demonstrating Streams and KSQL operating on JSON data.

The Gradle tasks streamJSONRatings and streamAvroRatings will run the JSONRatingStreamer and AvroRatingStreamer tasks, respectively. (A future refactoring should combine those two.) These classes generate randomized ratings on a subset of films, each with its own target average rating. This is handy for demos.

You can’t perform that action at this time.