Provide a kafka streams/event stores based event log #47
Feature request as per
This way, the concepts of "entity" from
The text was updated successfully, but these errors were encountered:
@SemanticBeeng that's the topic I thought a lot about. When you use Kafka as an event log you loose the ability to query events by entity key. Imagine the time you need to recover the whole system in case you need to change how you fold an entity state? Also, runtime state that uses KTable or similar is eventually consistent which is acceptable for read side, but not for write side.
Please help me understand better your points, especially "acceptable for read side, but not for write side.".
But for "ability to query events by entity key." we could use a separate topic per entity.
Again, please explain a bit deeper to see what I am missing since have not thought as much as you.
I think there are limitations to
Apache Pulsar is also an interesting middleware to explore as a target, it supports both a topic/pub-sub interface as well as a queue interface (suitable for work scheduling for example) and it scales to millions of topics, therefore it's conceivable you could have a topic for each host in your akka cluster processing and therefore if you're doing something similar to an Ask pattern you might be able to implement a direct (and durable) return to sender reply without finishing to be routed via akka remoting (but then you need to figure out what happens if the host which Asked something dies).
Apologies if this explains something obvious.
I also didn't understand what was meant by "acceptable for read side, but not for write side", in Lagom Read Side Processor designates a role processes the Events (in the Command (input)-> State (hidden) -> Event (output) sense) and then prepares some sort of a view which is query-able or more optimised for some sort of read pattern. Are you talking about the problem arising of re-processing Commands which requires some sort of strategy to de-dupe the already processed ones?