Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make a simple way to perform "the outbox pattern". #23

Open
stolsvik opened this issue Aug 12, 2019 · 1 comment
Open

Make a simple way to perform "the outbox pattern". #23

stolsvik opened this issue Aug 12, 2019 · 1 comment
Labels
enhancement New feature or request thoughts Issues describing some thoughts around a subject

Comments

@stolsvik
Copy link
Contributor

The outbox pattern is a way to overcome the rather bad scenario in messaging: You receive a message, start a DB transaction, perform some state changing operations on the DB, send an outgoing message, commit the DB - and when about to commit the MQ to both acknowledge the processing of the incoming message, and push the outgoing message onto the MQ, the MQ connection goes down, the MQ crashes, or the micro service crashes or whatever other nasty stuff affecting your commit of the MQ transaction.

The point is that you have now performed the state changing operation on the database, but you have neither acknowledged the receipt of the incoming message, nor had the outgoing message pushed to the MQ.

You will now most probably get a redelivery of the incoming message. But it should stop processing right there, as you have double-delivery catching in place. But that effectively means that the process has stopped.

The "outbox pattern" - which evidently have gotten lots of attention in 2019 - basically sticks the outgoing message on the DB as an intermediate step. The point is that it is then committed together with the DB-commit of the other state changing messages. Another process picks up the outgoing messages from the outbox and pushes them "onto the wire". This process can also crash, but it should crash in a way that ensures that the message will be delivered at least once. And since all messaging endpoints should either be idempotent, or have double-delivery catching, that won't be a problem.

One random link from Google: http://www.kamilgrzybek.com/design/the-outbox-pattern/

@stolsvik
Copy link
Contributor Author

stolsvik commented Jan 2, 2020

Make sure to expose this on the public API, so that I can use it in MatsSockets. That is, when initiating a message, instead of doing the actual send, request or publish, ask for a "outbox representation", returning a byte array that can be stored. This must be possible then to "post to outbox" (with or without backing db).

Could be a method "returnOutbox", which then gave the byte array as a part of the returned value from the send, request and publish methods.

@stolsvik stolsvik transferred this issue from another repository Sep 27, 2021
@stolsvik stolsvik added thoughts Issues describing some thoughts around a subject improvement Improve an existing feature enhancement New feature or request and removed improvement Improve an existing feature labels Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request thoughts Issues describing some thoughts around a subject
Projects
None yet
Development

No branches or pull requests

1 participant