pubsub.get_message with handler - how know if message has been processed #1865
Description
The description is that pubsub.get_message() returns None immediately if there are no messages to read and thus fits into the typical application event loop. Also any message taken care of by handler will return None. The described concept is this, simplified:
while True:
event = some.poll(timeout=0.1)
if event == EV1:
doEvent1()
pubsub.get_message() # Handlers will process, non-block if no messages
The application events can be processed at any speed they arrive. Redis messages however will be limited to 10 / s (when there are no application events that drives the loop faster). The latency of 0.1s here is not the problem but that this construct cannot catch up and process all redis messages that may be waiting.
If it where possible to add an inner while loop for get_message() until no more messages where handled we would catch up quickly.
But this is not possible since get_message() returns None both if there is no message at all or if a handler took care of it. Adding global variables in handlers to signal they been called would be messy. Only remaining alternative seem to not use handlers, and loop while we get something and manually dispatch to handlers. (Well this will not work completely as subscription messages and using ignore_subscribe_messages flag also returns None, but this would be a corner case).
It seems odd that there is no case to handle this properly, especially since it would have been easy to return something differentiating between a true TIMEOUT with nothing processed and no need to iterate on get_message(), None for no message to process (done in handler) and an actual message to be handled. Where last two cases would continue iterating get_message() until TIMEOUT.
Or, is there some other way one can handle this?