demo-scene

Scripts and samples to support Confluent Platform talks. May be rough around the edges. For automated tutorials and QA'd code, see https://github.com/confluentinc/examples/

Requirements

You need to allocate Docker 8GB when running these. Avoid allocating all your machine's cores to Docker as this may cause the machine to become unresponsive when running large stacks. On a four-core Mac Book two cores for Docker should be fine.

Contents

• Data pipelines (Database + KSQL + Elasticsearch)

  • Rail data streaming pipeline
  • MySQL / Debezium CDC / KSQL / Elasticsearch
  • Oracle CDC / KSQL / Elasticsearch
  • Postgres / Debezium CDC / KSQL / Elasticsearch
  • CDC demo with MySQL
  • CDC demo with Oracle
  • Building data pipelines with Confluent Cloud and GCP (BigQuery, GCS, etc)

• KSQL

  • An introduction to KSQL
  • Live-Coding KSQL scripts
  • KSQL UDF Advanced Example
  • KSQL Troubleshooting
  • ATM Fraud detection with Kafka and KSQL
  • Kafka Streams/KSQL Movie Demo
  • KSQL MQTT demo
  • KSQL Dump Utility

• Kafka Connect

  • MQTT Connect Connector Demo
  • Kafka Connect deepdive - understanding converters and serialization
  • Kafka Connect JDBC Source demo environment
  • Example Kafka Connect syslog configuration and Docker Compose

• Confluent Cloud

  • Pac-Man Demo
  • "The Cube" Demo

• Misc

  • Hacky export/import between Kafka clusters using kafkacat
  • Docker Compose for just the community licensed components of Confluent Platform
  • Topic Tailer, stream topics to the browser using websockets
  • KPay payment processing example
  • Industry themes (e.g. banking Next Best Offer)

Feedback & Questions

Raise an issue on this github project, or head to http://cnfl.io/slack and join our Community Slack group.