live-demo___python-jupyter-apache-kafka-ksql-tensorflow-keras.adoc

Live Demo: Python, Jupyter notebook, TensorFlow, Keras, Apache Kafka and KSQL

Kai Waehner <kontakt@kai-waehner.de> 29 Nov 2018

This script assumes that all components like Zookeeper, Kafka, KSQL, Jupyter) use default values.

Starting backend services

First we need to start a local Kafka ecosystem to use KSQL from the Jupyter notebook. We also start a generator to create a continous feed of streaming data.

// Start KSQL with Kafka and other dependencies
CONFLUENT_HOME# confluent start ksql-server

// Start data generator
CONFLUENT_HOME# ksql-datagen quickstart=users format=json topic=users maxInterval=1000  propertiesFile=etc/ksql/datagen.properties

Open Jupyter notebook

// Open Jupyter and select the notebook 'live-demo___python-jupyter-apache-kafka-ksql-tensorflow-keras.adoc'
jupyter notebook

Follow the steps in the notebook to run the demo.

Jupyter / pip / conda commands

Some common commands for Jupyter, pip, conda to manage packages like ksql-python:

pip info

pip install ksql

conda info

source activate ksql-python

conda install tensorflow

conda remove -n ksql-server scipy