This is a prototype for a POC - it is not production ready!
See this blog post for an introduction: https://zeebe.io/blog/2018/12/writing-an-apache-kafka-connector-for-zeebe/
- Correlate messages from a Kafka topic with Zeebe workflows. This uses the Zeebe Message Correlation features. So for example if no matching workflow instance is found, the message is buffered for its time-to-live (TTL) and then discarded. You could simply ingest all messages from a Kafka topic and check if they correlate to something in Zeebe.
- Send messages from a workflow in Zeebe to a Kafka topic.
How to use the connector
- Build via mvn package
- Put the resulting UBER jar into KAFKA_HOME/plugins
- Run Kafka Connect using the Connector pointing to the property files listed below: connect-standalone connect-standalone.properties zeebe-sink.properties zeebe-source.properties
Sink (Kafka => Zeebe)
The sink will forward all records on a Kafka topic to Zeebe (see sample-sink.properties):
name=ZeebeSinkConnector connector.class=...ZeebeSinkConnector correlationJsonPath=$.orderId messageNameJsonPath=$.eventType zeebeBrokerAddress=localhost:26500 topics=flowing-retail ...
In a workflow model you can wait for certain events by name (extracted from the payload by messageNameJsonPath):
Source (Zeebe => Kafka)
The source can send records to Kafka if a workflow instance flows through a certain activity (sample-source.properties):
name=ZeebeSourceConnector connector.class=...ZeebeSourceConnector zeebeBrokerAddress=localhost:26500 topics=flowing-retail
In a workflow you can then add a Service Task with the task type "sendMessage" which will create a record on the Kafka topic configured: