You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The idea was that when a DataFlowTask is spawned, it is added to the DAG but the actual scheduling of the underlying task is delegated to a worker task which takes nodes from a runnable priority queue and executes them. After a task is finished, it inserts itself in a finished queue for removal (by another worker).
Pushing tasks into the runnable queue happens in one of two ways:
A task inserts itself into runnable if it is dependency-free when it is created.
If a task has a dependency when created, it is the responsibility of its last unfinished dependency to put it in the runnable queue. This means that the cleanup phase of finishing a task involves
Removing itself as a dependency on all of its outgoing edges
If it was the last dependency of one of those outgoing nodes, add that node to runnable
Remove itself from the DAG
This was dropped for simplicity until the package matures a bit more, and because I was not very happy with the way tasks were executed under this scheduler, but perhaps it is worth bringing it back to live at some point since priority scheduling can help guide the scheduler through the critical path more easily.
The text was updated successfully, but these errors were encountered:
At some point, we may want to (add back?) support for a
PriorityScheduler
along the lines of what can be found inDataFlowTasks.jl/src/otherschedulers.jl
Line 108 in cf15cc7
The idea was that when a
DataFlowTask
is spawned, it is added to theDAG
but the actual scheduling of the underlying task is delegated to a worker task which takes nodes from arunnable
priority queue and executes them. After a task is finished, it inserts itself in afinished
queue for removal (by another worker).Pushing tasks into the
runnable
queue happens in one of two ways:runnable
if it is dependency-free when it is created.runnable
DAG
This was dropped for simplicity until the package matures a bit more, and because I was not very happy with the way tasks were executed under this scheduler, but perhaps it is worth bringing it back to live at some point since priority scheduling can help guide the scheduler through the critical path more easily.
The text was updated successfully, but these errors were encountered: