Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Support for "Task Dependencies" in BibSched #612

Closed
invenio-developers opened this Issue · 3 comments

5 participants

@invenio-developers
Collaborator

Originally by boltmann on 2011-04-19

For the new module handling videos in Invenio ("BibEncode") we need support for task dependencies in BibSched. There should be a parameter upon task submission that defines an execution dependency on another task that is currently running.

"BibEncode" has a daemon mode that launches multiple other "BibEncode" tasks for transcoding and other stuff defined by a submission.
Let us say that one of these tasks ("A") moves a video file and alters the metadata of the video.
Other "BibEncode" tasks ("B", "C") are going to trancode the video afterwards and extract thumbnails from it.
"B" and "C" cannot be executed before the videos was moved and altered by task "A".
On submission of "B" and "C" a parameter should be settable that means "do not start before task "A" has been executed successfully".

One possible way would be to have the task resubmit itself in case that the task it depends upon was not yet successfully executed.

This is different from the the "postprocess" approach because multiple tasks, possibly in parallel, need to be run.

@kaplun
Collaborator

Originally on 2011-04-19

Additionaly, when the task resubmit itself, it has to choose a time in the future. This could be 60s + max(now, execution time of the tasks it depends on).

In this way the task would be woken up at worst every minute.

@jirikuncar
Owner

@kaplun @egabancho @jalavik it seems that it's going to be superseded by workflows and celery tasks, am I right?

@jirikuncar jirikuncar added this to the v2.x milestone
@jalavik
Collaborator

Indeed workflows and celery will do this nicely.

@lnielsen lnielsen closed this
@jirikuncar jirikuncar modified the milestone: v2.2.0, v2.1.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.