Skip to content

Latest commit

 

History

History
 
 

06-streaming

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
 
 

Week 6: Stream Processing

Code structure

Confluent cloud setup

Confluent cloud provides a free 30 days trial for, you can signup here

Introduction to Stream Processing

  • Slides

  • 🎥 6.0.1 Introduction

  • 🎥 6.0.2 What is stream processing

Introduction to Kafka

  • 🎥 6.3 What is kafka?

  • 🎥 6.4 Confluent cloud

  • 🎥 6.5 Kafka producer consumer

Kafka Configuration

  • 🎥 6.6 Kafka configuration

Kafka Streams

  • 🎥 6.8 Kafka stream join

  • 🎥 6.9 Kafka stream testing

  • 🎥 6.10 Kafka stream windowing

  • 🎥 6.11 Kafka ksqldb & Connect

  • 🎥 6.12 Kafka Schema registry

Faust - Python Stream Processing

Pyspark - Structured Streaming

Please follow the steps described under pyspark-streaming

  • 🎥 6.13 Kafka Streaming with Python

  • 🎥 6.14 Pyspark Structured Streaming

Kafka Streams with JVM library

KSQL and ksqlDB

Kafka Connect

Docker

Starting cluster

Command line for Kafka

Create topic

./bin/kafka-topics.sh --create --topic demo_1 --bootstrap-server localhost:9092 --partitions 2

Homework

Community notes

Did you take notes? You can share them here.