Skip to content

Commit 9850928

Browse files
committed
Described steps of the demo and (upcoming) Notebook content
1 parent 2f6f3fb commit 9850928

File tree

1 file changed

+15
-2
lines changed

1 file changed

+15
-2
lines changed

readme.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,18 @@ Building a demo which combines
1515

1616
## Step-by-step guide
1717

18-
Check out [these steps](https://github.com/kaiwaehner/python-jupyter-apache-kafka-ksql-tensorflow-keras/blob/master/live-demo___python-jupyter-apache-kafka-ksql-tensorflow-keras.adoc) to start the backend and notebook. The main demo is running in the Jupyter notebook then.
18+
We will do the following:
19+
20+
1) Data Integration (Kafka Connect): Integrate a stream of data from CSV file or continuous data stream (in real world you can connect directly to an existing Kafka stream from the Jupyter notebook)
21+
2) Data Preprocessing (KSQL): Preprocess the data, e.g. filter, anonymize, aggreate / concatenate
22+
3) ML-specific preprocessing (NumPy, Scikit-learn): Normalize, split train / test data
23+
4) Train model (TensorFlow + Keras)
24+
5) Deploy model (KSQL + Tensorflow)
25+
6) Monitor model inference (KSQL)
26+
27+
While all of this can be done in a Jupyter notebook for interactive analysis, we can then deploy the same pipeline to production at scale. For instance, you can re-use the KSQL preprocessing statements and run them in your production infrastructure to to model inference with KSQL and the TensorFlow model at scale.
28+
29+
Check out [this document](https://github.com/kaiwaehner/python-jupyter-apache-kafka-ksql-tensorflow-keras/blob/master/live-demo___python-jupyter-apache-kafka-ksql-tensorflow-keras.adoc) to start the backend and notebook. The main demo is running in the Jupyter notebook then and shows all above steps.
1930

2031

2132
## Autoencoder for Credit Card Fraud Detection build with Keras and TensorFlow
@@ -50,4 +61,6 @@ The Python tool [Keras to TensorFlow](https://github.com/amir-abdi/keras_to_tens
5061

5162
python keras_to_tensorflow.py --input_model="/Users/kai.waehner/git-projects/python-jupyter-apache-kafka-ksql-tensorflow-keras/models/autoencoder_fraud.h5" --output_model="/Users/kai.waehner/git-projects/python-jupyter-apache-kafka-ksql-tensorflow-keras/models/autoencoder_fraud.pb"
5263

53-
The tool freezes the nodes (converts all TF variables to TF constants), and saves the inference graph and weights into a binary protobuf (.pb) file.
64+
The tool freezes the nodes (converts all TF variables to TF constants), and saves the inference graph and weights into a binary protobuf (.pb) file.
65+
66+
TODO Use keras.estimator.model_to_estimator (included in tf.keras)? Example: https://www.kaggle.com/yufengg/emnist-gpu-keras-to-tf

0 commit comments

Comments
 (0)