Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
API for provisioning persistent stream subscriptions #86
We could provide an API for provisioning persistent stream subscriptions between services. By persistent, I mean if a disconnect occurs, it reconnects (with exponential backoff).
The purpose of this is to provide point to point at least once message delivery between two clustered services.
At a minimum we could support running as a cluster singleton, but we could also let Lagom handle partitioning, so that we can subscribe to partitioned streams - this would need to be paired with nice partitioning support in the persistent entities event stream (probably isn't any hard then simply tagging events by partition, sharding by event id over the number of partitions). To implement this, we could simply allow the client side to shard partitions across the cluster - it would need to ask the service how many partitions there are to do so.
Additionally, we could also provide some helpers that would manage recording and loading offsets in Cassandra for implementing at least one delivery - this doesn't have to be part of the persistent stream API, but could be a helper component that helps in implementing the code that sets up the stream.