Skip to content

TransactionalStreamDataBatchProcessing

James Hu edited this page Aug 27, 2015 · 6 revisions

This is a class utilizing SequentialTransactionsCoordinator and StreamDataSupplier to process the data stream. The purpose of this class is to create [something equivalent to .Net EventProcessorHost in the Java world](Java equivalent of EventProcessorHost for processing events in Azure Event Hub).

Its functionalities are actually beyond what EventProcessorHost can offer.

  • It is flexible. It supports variable batch size and allows batch size and event receive/fetch time out to be controlled directly by code.
  • It is generic. By implementing StreamDataSupplier interface, you can have it support other event queues like AWS Kinesis and Apache Kafka.
  • It has a more generalized model. Rather than requiring one check point and one in-progress batch per stream/partition, TransactionalStreamDataBatchProcessing allows you to have multiple batches being processed by different threads/computers at the same time.
  • It is transactional - once the range of data in a batch is determined, it won't change. All the data within this range will always have the same state.

For how to use, see [this page](Java equivalent of EventProcessorHost for processing events in Azure Event Hub).