You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, this one slipped for many reasons - one being I'm not a regular/savvy user of Redis. One solution would be to implement a future backend on top of the rrqueue package, which builds upon Redis.
The very rough outline would be to create a RrqueueFuture class that extends the Future class (or possibly the MultiprocessFuture class) and implements a run() that launch/spawn to rrqueue, a resolved() that checks with rrqueue whether the task is done, and a value() that collects the value from rrqueue. It looks like rrqueue already has functions for those mechanisms, so it could be that a first prototype would only require a very shallow set of wrappers.
Hi @HenrikBengtsson - we're in the process of tidying up the successor to rrqueue at the moment (https://github.com/mrc-ide/rrq/) which will go to CRAN later in the year. Where would one start looking for implementing a future backend, and I can either look at this or see if I can put pieces in place to make it straightforward
I'd be interested in learning more on Redis and how it can be used for orchestrating parallel processing, and would be willing to help with the future end of the task. I know you've used it for a long time. We could start with a call to synchronize expectations on how futures and Redis can work together. I'll ping you by email.