This may be nitpicking, but IMHO we should only ack the message after all postprocessing was done.
I understand that this is arguable, since currently we ack after we saved the raw data. I just feel this to be more correct, even if we do not check the success of the postprocessing. Ideally we should wrap both in a transaction and only ack on successful commit.
Feel free to close this if you don't agree.
only ack message after all (postprocessing) work is done
Yeah, this is a tricky one that we've debated quite a bit.
What happens in practice is that 90% of problems are in the post-processing stage and, in large scale production, this can mean the rabbit queues exploding. It's better to ack if we're at least written the notification to disk. Ideally, the post-processing should be idempotent so we could redo them if needed.
So, for now, we have to leave it the old way. We're looking at ways to improve the post-processing to better handle this.
BTW> Thanks for all the PR's! Great to have more eyeballs on the projects!