Ack only after processing #286

Closed
wants to merge 1 commit into
from

Projects

None yet

2 participants

@bkw
Contributor
bkw commented Feb 6, 2014

This may be nitpicking, but IMHO we should only ack the message after all postprocessing was done.
I understand that this is arguable, since currently we ack after we saved the raw data. I just feel this to be more correct, even if we do not check the success of the postprocessing. Ideally we should wrap both in a transaction and only ack on successful commit.

Feel free to close this if you don't agree.

Contributor

Yeah, this is a tricky one that we've debated quite a bit.

What happens in practice is that 90% of problems are in the post-processing stage and, in large scale production, this can mean the rabbit queues exploding. It's better to ack if we're at least written the notification to disk. Ideally, the post-processing should be idempotent so we could redo them if needed.

So, for now, we have to leave it the old way. We're looking at ways to improve the post-processing to better handle this.

@SandyWalsh SandyWalsh closed this Feb 6, 2014
Contributor

BTW> Thanks for all the PR's! Great to have more eyeballs on the projects!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment