Ensure transaction safety #159
Comments
This centralizes the invokation of `when*()` event handlers to a central
class that ensures the following invokation order:
# `beforeInvokingEventListenerMethod()` (when implementing `ActsBeforeInvokingEventListenerMethodsInterface`)
# `when*()` (the actual event handler)
# `saveHighestAppliedSequenceNumber()` (when implementing `AsynchronousEventListenerInterface`)
# `afterInvokingEventListenerMethod()` (when implementing `ActsAfterInvokingEventListenerMethodsInterface`)
This allows EventListeners to wrap the event handling in transactions for example:
```
class SomeEventListener implements AsynchronousEventListenerInterface, ActsBeforeInvokingEventListenerMethodsInterface, ActsAfterInvokingEventListenerMethodsInterface
{
public function beforeInvokingEventListenerMethod(EventInterface $event): void
{
// start transaction
}
public function whenSomeEvent(SomeEvent $event): void
{
// db interaction
}
public function saveHighestAppliedSequenceNumber(int $sequenceNumber): void
{
// update sequence number in db
}
public function afterInvokingEventListenerMethod(EventInterface $event): void
{
// commit transaction
}
}
```
This change...
* Renames `EventListenerLocator` to `EventListenerManager` – the class
is now also responsible for *invoking* event listeners
* Removes `EventPublisher::catchUp()` and moves the remaining logic to
the `EventCommandController`
* Introduces an `ActsAfterInvokingEventListenerMethodsInterface` marker
interface, that is used similarily to `ActsBeforeInvokingEventListenerMethodsInterface`
and allows to wrap event handling in a transaction
* Introduces a new `EventCantBeAppliedException` that allows for easier handling of
exceptions during event handling (see neos#161)
* Removes an unused `projectionCache` configuration
Related: neos#159
bwaidelich
changed the title
|
I renamed this ticket from "Wrap ProcessManager lastAppliedSequenceNumber and state update into a single transaction" in order to turn it into an umbrella ticket for the transaction-related issues: EventListener last applied event & state updates
Prevent multiple events with the same versionLuckily that is already guaranteed because of the unique key constraint in the events table. But we might want to wrap the corresponding exception (if two processes try to publish to the same stream "simultaneously") so that is easier to deal with – even though the risk of that is probably quite low Exactly-once-deliveryWe can't guarantee exactly once delivery (e.g. in ProcessManagers). We might be able to increase the chances that an event is processed exactly once using some kind of queue, but that is out of scope for this ticket |
|
I didn't get as far with this as I planned originally, but I spend some more thoughts on this: The following activity diagrams are hard to read, but I can use them as memory aid for further discussions: Note: HSN = Highest Sequence Number Catchup all events per Handler in one transaction:
One transaction per event:
Questions:
|
|
I'd like to dig into this topic again, and I tried twice already, but it's really deep and I'm a bit out of it. I hope to find some time for deep diving into this sometime in the near future, but for the moment, that won't be possible. |
|
@albe Thanks for your comment! The list above was more of a mental aid in order to be able to discuss this with others. In the meantime I have a working(tm) solution that allows multiple workers to work in parallel without blocking / re-applying the same event twice. But it needs cleanup and a better architecture.. I tend to create a new branch so that I can push my intermediate state more often... Would be cool to have your input from time to time, it is greatly appreciated! |
Resolves: neos#159
|
There are still some related issues (#245 for example), but the "AppliedEventsStorage" mechanism should be safe now! |
See #141 (comment)
Wrap ProcessManager lastAppliedSequenceNumber and state update into a single transaction
Without the two happening within a transaction, we potentially apply state transitions inside the PM multiple times.
Case:
event X comes in
PM updates state
crashes before updating lastAppliedSequenceNumber
-> catch up from the same event and reapply it after restart
The text was updated successfully, but these errors were encountered: