add support to continue tasks #411

Closed
kaazoo opened this Issue May 4, 2011 · 3 comments

Projects

None yet

2 participants

@kaazoo
kaazoo commented May 4, 2011

With this example from minrk it is possible to requeue a stopped given task:

from IPython.parallel import Client
from IPython.parallel.utils import unpack_apply_message

rc = Client()
view = rc.load_balanced_view()

# get the buffers from the Hub
rec = rc.db_query(dict(msg_id=msg_id), keys=['buffers'])[0]
# reconstruct the arguments
f,args,kwargs = unpack_apply_message(rec['buffers'])
# resubmit
view.apply_async(f,*args,**kwargs)

However this will create a new task because it submits a copy with the same contents. If you don't want to keep old stopped tasks after you requeued them, you'll have to delete them with purge_results(msg_id).

Is there a way to keep the old tasks and just set their status to 'pending' again?

@minrk
Member
minrk commented May 4, 2011

true task resubmision is definitely on my todo list, but not yet implemented.

That resubmit code I showed you is not meant to be the real solution, just a hack that sort of works in the meantime.

@minrk
Member
minrk commented May 4, 2011

I went ahead and implemented a resubmit method as part of #413. If you check out that branch, you can see if it provides what you need. Pass one or more msg_ids, and the task(s) will be re-run from the Hub. No client-level db shenanigans necessary.

@kaazoo
kaazoo commented May 5, 2011

Thanks for implementing this funtionality. I'll add further feedback at #413.

@kaazoo kaazoo closed this May 5, 2011
@minrk minrk added a commit to minrk/ipython that referenced this issue May 17, 2011
@minrk minrk add Client.resubmit for re-running tasks
closes gh-411

* allow `content` in session.serialize to be a unicode object, because mongo+JSON cannot be relied upon to produce encoded bytes.
0c043a6
@mattvonrocketstein mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014
@minrk minrk add Client.resubmit for re-running tasks
closes gh-411

* allow `content` in session.serialize to be a unicode object, because mongo+JSON cannot be relied upon to produce encoded bytes.
66448fc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment