Universal trigger handling

[hjoliver]

This can probably be consolidated with several related/intertwined issues in due course:

• Improve interaction between task state, output and prerequisite #2329
• External triggering plugins? #2339
• Follow up on persistent broker state  #1902
• (others?)

Currently we treat several kinds of task trigger separately:

• prerequisites on other tasks - via the "broker" that records outputs to match with un-met prerequisites
• old-style external triggers (via the comms layer and cylc ext-trigger)
• clock-triggers - these are built in to task proxies, even though they are really a form of external trigger.
• (new-style external trigger plugins External triggering plugins? #2339 - in progress - where the suite daemon repeatedly checks some external system)

All of these can probably be handled in the same way. Each task has a bunch of triggers that express dependence on "something else". The "something elses" generate outputs that once completed can be expressed as strings to match triggers/prerequisites. These could all be stored in the one data structure and used to satisfy unsatisfied triggers in task proxies.

Before we can do this, we have to manage persistence of external trigger "outputs". Task outputs are currently persistent in memory because we keep succeeded task proxies in the pool, and on disk because task proxy state is stored in the DB (used to recreate task proxies at at restart). External triggers are not represented by in-memory objects like a task proxy. Some external triggers may be externally persistent - e.g. presence of a file on disk could be checked and rechecked as needed. But some may not be - e.g. a message from a message queueing system may disappear from the queue once consumed by the suite daemon. Therefore, generally, the suite daemon needs to store external outputs as well as task outputs. This fits nicely with our plans to retain task outputs rather than full task proxies, after jobs have completed. The in-memory outputs store could retain task and external outputs equally, with a DB table for persistence too.

[hjoliver]

A bit more detail: task proxies will have a list of prerequisite strings that can be satisfied by matching "outputs". Task proxies do not need to know where those outputs come from. The trigger definitions determine how the suite daemon can get the outputs to be generated:

• inter-task triggers, defined by the graph: run the task jobs, which generate output messages that will end up in the outputs data structure to directly satisfy task prerequisites
• external triggers (e.g. on an external file, or a message in and external message queuing system, or on a wall clock time, etc.): the trigger definition determines what the suite daemon does, periodically, to see if the output has been generated; when it has, put it in the same outputs data structure to directly satisfy task prerequisites.