-
Notifications
You must be signed in to change notification settings - Fork 3
TransactionalStreamDataBatchProcessing
James Hu edited this page Aug 27, 2015
·
6 revisions
This is a class utilizing SequentialTransactionsCoordinator and StreamDataSupplier to process the data stream. The purpose of this class is to create [something equivalent to .Net EventProcessorHost in the Java world](Java equivalent of EventProcessorHost for processing events in Azure Event Hub).
Its functionalities are actually beyond what EventProcessorHost
can offer.
- It is flexible. It supports variable batch size and allows batch size and event receive/fetch time out to be controlled directly by code.
- It is generic. By implementing
StreamDataSupplier
interface, you can have it support other event queues like AWS Kinesis and Apache Kafka. - It has a more generalized model. Rather than requiring one check point and one in-progress batch per stream/partition,
TransactionalStreamDataBatchProcessing
allows you to have multiple batches being processed by different threads/computers at the same time. - It is transactional - once the range of data in a batch is determined, it won't change. All the data within this range will always have the same state.
For how to use, see [this page](Java equivalent of EventProcessorHost for processing events in Azure Event Hub).