Tasks can force foreground execution in the LocalDispatcher

[andrewkaufman]

Now that LocalDispatcher has an executeInBackground option, some of the tasks may need to force foreground execution (things that need doing on the main thread of the live app for example).

• LocalDispatcher should add an executeInForeground plug to ExecutableNodes during setupPlugs(), which overrides it's own executeInBackground plug.
• Tasks executed in the foreground use ExecutableNode::execute() directly rather than the gaffer execute app.
• Any node with executeInForeground on forces all of its requirements to execute in the foreground as well.
• All tasks use ExecutableNode::execute() directly when executeInBackground is off.

[johnhaddon]

Do you think there's a benefit to this being a user-facing option? I've been trying to think of use cases outside of a hosted app, and struggled a little bit. The best I can come up with is if you know that you have some tasks which could execute much more quickly in the foreground, because they're already in the cache, whereas they'd have to be recomputed if done in the background.

In the use case that motivates this, would we ever want the user to turn executeInForeground off? If not, perhaps it's worth registering a null PlugValueWidget creator to hide it from the UI?

[andrewkaufman]

I'll have a think about it, though I think your user-knows-whats-quicker example is a pretty good one. On the particular nodes we're talking about in the hosted app, I'd imagine they'd either hide or lock it since it needs to be True.

[andrewkaufman]

Regarding the 3rd tick box "Any node with executeInForeground on forces all of its requirements to execute in the foreground as well."

The simplest implementation of this isn't ideal. My idea was to look at the TaskDescriptions and find the last one with an executeInForeground node. All tasks before that would be in the foreground, and all after would be in the background. In a complex setup this can result foreground execution for several tasks that could have run in the background if we did something smarter. This happens because some of the tasks from a foreground node appear in the TaskDescriptions after some of the tasks from a background node (there is no requirement between the 2 nodes).

Do you have any ideas regarding this? Did I even explain that in a way that makes sense to anyone but me?

[johnhaddon]

Yeah, it's almost like you need some sort of tree representation, so as you walk the tree from the root you can turn on the "executeInForeground" flag when you hit one of those nodes, and that flag is then on for any requirements that you subsequently recurse to. I wonder if we have a design for some sort of tree of batches of tasks knocking around anywhere?

[andrewkaufman]

Hardee-har-har. I guess I'll get on that redesign then...