Add to possibility to ensure repeated jobs are there

[mxzinke]

In case the Redis instance is crashing or getting flushed, all jobs are gone. It would be nice to have an option to ensure at least the repeatable jobs are still there.

This option could be included in the RepeatOptions and would just be a ensure property which could be a boolean. The queue then holds the already added jobs (with the ensure option set) in memory and tells the Redis instance to reschedule a job, when the connection has been lost or a flush event occurs.

The only issue which I found, is that the clean and remove functions for removing repeatable jobs, have to still make it possible. This also means that we have to publish a remove-event over a Redis channel so that on all instances the repeatable job is cleaned up out of the memory. A lot of overhead for a small feature, but I don't know the current state of implementation and can't value how expensive it would to implement.

Another possibility would be to have the ability to execute a function on Redis reconnecting. Which would be way easier to implement. But on this, the developers have to implement a logic themself to come back to the last state of the jobs (for example when using the limit option on the repeatable jobs).

Please share your thoughts about it.

[manast]

If I understand your proposal you mean to add like a cache layer on top of redis to handle fatal redis scenarios. If that is the case then I would suggest making sure redis can handle such a scenario, either by persisting to disk or by having redundant replicas that can take over in the case the primary instance crashes. Building a recovery logic on top of redis would be really complex and I do not see a lot of benefits considering you can handle it at the redis level.