Feature Request: add Process() method, eg Pop() an item without removing it but keep it hibernate for a period #996

Closed
diyism opened this Issue Mar 11, 2013 · 1 comment

1 participant

@diyism

Is there any method in Redis to pop an item without removing it but keep it hibernate in a period? After the period (and the item was not be processed correctly and haven't be deleted), this item wake up and can pop again.

Use case:
In my company, i receive huge amount of orders from many online shops and push it into Redis list, then pop items from Redis list for slow process. But if the slow process failed, the items lost.
I think the best solution is to add Process() method to Redis list, when i use Process() to pop an item, it will be set hibernate in a period, if the slow process successfully processed it, the slow process will call Delete() to remove it, if the slow process failed unexpected, the item will revive.
It's a big convenience.

References:
http://stackoverflow.com/questions/10882713/redis-list-pop-without-removing/15288077#15288077

@diyism

http://redis.io/commands/rpoplpush

Pattern: Reliable queue
Redis is often used as a messaging server to implement processing of background jobs or other kinds of messaging tasks. A simple form of queue is often obtained pushing values into a list in the producer side, and waiting for this values in the consumer side using RPOP (using polling), or BRPOP if the client is better served by a blocking operation.
However in this context the obtained queue is not reliable as messages can be lost, for example in the case there is a network problem or if the consumer crashes just after the message is received but it is still to process.
RPOPLPUSH (or BRPOPLPUSH for the blocking variant) offers a way to avoid this problem: the consumer fetches the message and at the same time pushes it into a processing list. It will use the LREM command in order to remove the message from the processing list once the message has been processed.
An additional client may monitor the processing list for items that remain there for too much time, and will push those timed out items into the queue again if needed.

@diyism diyism closed this Apr 24, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment