Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

254 / Save and publish events atomically #268

Closed

Conversation

maxekman
Copy link
Member

@maxekman maxekman commented Oct 8, 2020

Description

Save and publish events atomically by wrapping them in a transaction.

Affected Components

  • Event Bus (both memory and MongoDB)
  • Event Store (both memory and MongoDB)

Related Issues

Fixes #254.

Solution and Design

Uses MongoDB (and in-memory simulated) transactions to only commit events to the event store after publishing has been successful.

Steps to test and verify

@maxekman
Copy link
Member Author

maxekman commented Oct 8, 2020

@maarek and @giautm Any input on this approach?

It does come with some downsides:

  • Requires MongoDB to be configured as a replica set (which it should for production anyway).
  • Requires MongoDB 4.4 to support creating collections inside transactions.

An additional benefit with having a transaction for the event store is that all of storing the events, projecting and publishing can eventually happen inside it. I believe this would also work for other DB backends (PostgreSQL for example).

@giautm
Copy link

giautm commented Oct 9, 2020

Hi, sorry for delay.

I don't think move publish progress to transaction is the solution. Because It cause new two problems:

  • If the publish progress too long - network delay for example, then the database will lock until the publish success/failed.
  • In case the publish success, commit transaction maybe failure. Then, other services will receive the "ghost-events" (Events that's not happened on the source).

@maxekman
Copy link
Member Author

Thanks @giautm for the input. I believe you are correct! Maybe some kind of outbox pattern is the way to go then..

@maxekman maxekman changed the title ISSUE-254 / Save and publish events atomically 254 / Save and publish events atomically Feb 14, 2021
Base automatically changed from master to main March 8, 2021 09:41
@maxekman maxekman force-pushed the ISSUE-254/save-and-publish-atomically branch 4 times, most recently from 874867b to 9eb03dd Compare March 29, 2021 11:49
@maxekman maxekman force-pushed the ISSUE-254/save-and-publish-atomically branch from 9eb03dd to febc765 Compare March 29, 2021 12:29
@maxekman
Copy link
Member Author

maxekman commented Sep 5, 2021

Closing until further work has been done on this (probably the outbox pattern).

@maxekman maxekman closed this Sep 5, 2021
@maxekman maxekman deleted the ISSUE-254/save-and-publish-atomically branch November 9, 2021 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Save & publish event atomically
2 participants