Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

apply event sourcing to message subscriptions #6920

Closed
wants to merge 8 commits into from

Conversation

pihme
Copy link
Contributor

@pihme pihme commented Apr 30, 2021

Description

This applies event sourcing to message subscriptions.

This implementation largely reflects option 2 described in the ticket.

It does so by defining an interface for the transient state. With this interface in place the following dependencies are introduced:

  • Pending...SubscriptionChecker now depends on the transient state interface (and no longer on mutable persistent state interface
  • the implementation of the transient state depends on the immutable persistent state (for lookup of subscriptions)

The implementation of the transient state is based on a simple sorted set.

Furthermore sentTime fields are removed from persistent state altogether.

Major Flaws

The transient state does not participate in transaction rollback. As such the transient state can get out of sync with persistent state. In such a scenario, a command could be resent ad infinitum, because it will never be removed from transient state.

Other Shortcomings

  • there is no upper cap of in memory storage
  • on restart it requires a full table scan to restore the in memory state
  • the commandSentTime is not always updated exactly when the command is sent, it is sometimes updated when the persistent state is changed (which is a good proxy, but not precise)

Related issues

closes #6364

Definition of Done

Code changes:

  • The changes are backwards compatibility with previous versions
  • If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/0.25) to the PR, in case that fails you need to create backports manually.

Testing:

  • There are unit/integration tests that verify all acceptance criterias of the issue
  • New tests are written to ensure backwards compatibility with further versions
  • The behavior is tested manually
  • The change has been verified by a QA run
  • The impact of the changes is verified by a benchmark

Documentation:

  • The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
  • New content is added to the release announcement

@camunda-cloud-bot
Copy link

@pihme: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

pihme added 8 commits May 19, 2021 14:26
This commit also contains some automatic reformatting applied
- also remove sentTime information from subescription state
- reduce and rename signatures of new interface
replace sentTime field in subscription state with correlating flag
- reduce visibility of methods
- improve thread safety
This is a reluctant change. Initially I was hoping to find a way to update the sent time
when the command is actually being sent, instead of using the time when the persistent
update is made. However, updating the command sent time at the point when the command is
sent is a bigger undertaking. Commands can be sent by different classes, none of which
have the transient state immediately available.
@pihme pihme force-pushed the 6364-message-subscriptions branch from 83a5180 to ffc4773 Compare May 19, 2021 15:19
@pihme pihme closed this May 19, 2021
@github-actions
Copy link
Contributor

Only merged pull requests can be backported.

@pihme
Copy link
Contributor Author

pihme commented May 19, 2021

PR will be closed and reopened once work on the issue has advanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Apply event-sourcing to message subscription retry
2 participants