BibSched: start tasks at fixed times

[tiborsimko]

Originally on 2011-08-29

The goal of this ticket is to enable some tasks to run at fixed times whatever the status of the queue is. A typical example is oaiharvest task that we may want to run every day at 04:01 sharp regardless of the status of the queue. There are two reasons behind this: (i) the remote OAI server we are getting data from publishes its feeds at 04:00 sharp and we want to get the data as soon as possible; (ii) the oaiharvest task is not conflicting with any other bibsched task that may be running at the moment and so is always safe to run at a concrete time whatever the status of the queue is. Provided that yesterday's harvesting jobs finished; IOW, oaiharvest is not sensitive to any other task but itself.)

We need to introduce a new variable to bibtask_config.py named something like CFG_BIBTASK_FIXED_TIME_TASKS that will lists all the tasks that are available to be run at fixed times, and bibsched will run them whatever the status of the queue is.

(Note that this depends a bit on the parallel task execution facility, see CFG_BIBSCHED_MAX_NUMBER_CONCURRENT_TASKS. This has been revived recently and shown to work. Note also that task dependencies may be populated more extensively later. For the time being, for the purpose of this ticket, it is sufficient to assume that a fixed time task can run when no other fixed time task is running.)

[invenio-developers]

Originally by vvenkatr on 2011-09-12

Bibsched : 794_bibsched_fixedtasks_fix

Adds configurable variable for fixed-time tasks and adds conditional checks in bibsched module for the same.

[invenio-developers]

Originally by Vasanth Venkatraman vasanth.venkatraman@cern.ch on 2012-02-02

In [ec246fe]:

#CommitTicketReference repository="" revision="ec246feb905e867e245b4898a2092fa1c6172af5"
BibSched: facility to start tasks at fixed time

- Adds new facility that permits to start certain tasks (such as
  oaiharvest) at fixed time, regardless of the status of the queue.
  (closes #794)

[invenio-developers]

Originally by Vasanth Venkatraman vasanth.venkatraman@cern.ch on 2012-08-09

In ec246fe:

#CommitTicketReference repository="" revision="ec246feb905e867e245b4898a2092fa1c6172af5"
BibSched: facility to start tasks at fixed time

- Adds new facility that permits to start certain tasks (such as
  oaiharvest) at fixed time, regardless of the status of the queue.
  (closes #794)

[invenio-developers]

Originally by Vasanth Venkatraman vasanth.venkatraman@cern.ch on 2012-08-09

In ec246fe:

#CommitTicketReference repository="" revision="ec246feb905e867e245b4898a2092fa1c6172af5"
BibSched: facility to start tasks at fixed time

- Adds new facility that permits to start certain tasks (such as
  oaiharvest) at fixed time, regardless of the status of the queue.
  (closes #794)