Kafka Connector for Zeebe.io (Prototype!)
Clone or download
Latest commit 228142f Dec 13, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src adjusted log level and some last cleanups Dec 12, 2018
windows-cmd adjusted log level and some last cleanups Dec 12, 2018
.gitignore switched to Zeebe 0.12.0-alpha4 and adjusted package name Sep 24, 2018
LICENSE Initial commit Aug 14, 2018
README.md Update README.md Dec 13, 2018
bpmn0.png added readme Dec 12, 2018
bpmn1.png added readme Dec 12, 2018
bpmn2.png added readme Dec 12, 2018
overview.png added readme Dec 12, 2018
pom.xml upgraded to Zeebe 0.14.0 and cleaned up a bit Dec 12, 2018

README.md

kafka-connect-zeebe

Kafka Connector for Zeebe.io

This is a prototype for a POC - it is not production ready!

See this blog post for an introduction: https://zeebe.io/blog/2018/12/writing-an-apache-kafka-connector-for-zeebe/

Features:

  • Correlate messages from a Kafka topic with Zeebe workflows. This uses the Zeebe Message Correlation features. So for example if no matching workflow instance is found, the message is buffered for its time-to-live (TTL) and then discarded. You could simply ingest all messages from a Kafka topic and check if they correlate to something in Zeebe.
  • Send messages from a workflow in Zeebe to a Kafka topic.

Overview

How to use the connector

  • Build via mvn package
  • Put the resulting UBER jar into KAFKA_HOME/plugins
  • Run Kafka Connect using the Connector pointing to the property files listed below: connect-standalone connect-standalone.properties zeebe-sink.properties zeebe-source.properties

Sink (Kafka => Zeebe)

The sink will forward all records on a Kafka topic to Zeebe (see sample-sink.properties):

name=ZeebeSinkConnector
connector.class=...ZeebeSinkConnector

correlationJsonPath=$.orderId
messageNameJsonPath=$.eventType

zeebeBrokerAddress=localhost:26500

topics=flowing-retail
...

In a workflow model you can wait for certain events by name (extracted from the payload by messageNameJsonPath):

Overview

Source (Zeebe => Kafka)

The source can send records to Kafka if a workflow instance flows through a certain activity (sample-source.properties):

name=ZeebeSourceConnector

connector.class=...ZeebeSourceConnector
zeebeBrokerAddress=localhost:26500

topics=flowing-retail

In a workflow you can then add a Service Task with the task type "sendMessage" which will create a record on the Kafka topic configured:

Overview