Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk processing #111

Closed
Dadibom opened this issue Oct 14, 2020 · 2 comments
Closed

Bulk processing #111

Dadibom opened this issue Oct 14, 2020 · 2 comments

Comments

@Dadibom
Copy link

Dadibom commented Oct 14, 2020

In many cases, processing batches of messages instead of processing them one at a time can provide huge performance benefits, such as multiple database inserts being able to be wrapped in a single transaction.

Having a simple subscribe method that provides a list of messages in the callback whenever there's multiple received messages at once would be great.

@gdaws
Copy link
Owner

gdaws commented Oct 26, 2020

Bulk processing of messages can be achieved at the application level. Open a subscription with client ack mode and when a message is received push it onto a local queue for the next processing batch. After a batch has been processed send an acknowledgement for the last message processed. The client sees the performance benefit of sending one ack for many messages and the server could potentially perform ack handling of many messages as a single transaction.

I'm not sure it's possible for stompit, by its design, to deliver a list of messages to a subscription callback. How can it know when multiple messages are received at once? The library only reads a frame header then control is passed to the subscription callback where that reads the frame body. It's for the application to read a message and then either process it immediately or defer it.

@Dadibom
Copy link
Author

Dadibom commented Dec 8, 2020

Thanks @gdaws, I don't even know if stompit polls or gets data pushed to it so I have no idea how it would be done internally. I'll just write a simple wrapper function and use that as a handler.

@Dadibom Dadibom closed this as completed Dec 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants