Use assign/pause/redirect/unpause pattern

[AlexeyRaga]

Changes

• Implement assign/pause/redirect/unpause pattern mentioned in Redirect partition queues *after* reassignment. #107 (comment)

Questions:

• does it address Redirect partition queues *after* reassignment. #107 ?
• does it eliminate the need of an ugly sleep to prevent unexplained jumps?'

@shlevy do you think this may work?

[AlexeyRaga]

I am still very skeptical about this because I don't see how it can be guaranteed that no messages can be delivered into a "default" consumer queue in between of assign and pause.

Or does the fact that we are already in an event loop safe us here? Like, even if some messages have been delivered to the "events" queue, pause will make sure that they are gone and they will be re-delivered to the appropriate queue after unpause happens?

[shlevy]

@AlexeyRaga Two thoughts:

• Does assign cause an unpause? If not can't we pause before assign?
• Maybe we need to pause our poll thread here?

I think we'll need @edenhill to reach any confidence here 😦

[edenhill]

If you don't have a separate thread that calls poll() and feeds messages into a local queue then it is perfectly fine to do assign() + pause(), since even if messages were fetched after the assign they will be purged by pause() and there is no poll done between those two calls.

[shlevy]

OK, so in that case @AlexeyRaga I think we need to do something like:

1. Have pollConsumerEvents take an mvar before polling and restore it afterward
2. Take the mvar in assignmennt
3. assign
4. pause
5. redirect
6. resume
7. restore mvar

[AlexeyRaga]

I now use an MVar to make sure that only one thread can poll for events/callbacks at the time.
I was able to finally remove that stupid thread delay and I cannot reproduce any jumps or double-dispatches.

@shlevy it would be great to check that for that double-dispatch issue that you had against this version.

[shlevy]

@AlexeyRaga Sounds good, I'll give this a shot this week.