Allow flows to set run state from "key_tasks" #95
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Check not only for cycles, but also for edges / key_tasks that reference non-existant tasks
jlowin
force-pushed
the
flow_state_from_tasks
branch
from
be63509
to
1b388c1
Compare
cicdw
requested changes
Aug 7, 2018
src/prefect/core/flow.py
Outdated
| @@ -58,6 +58,7 @@ def __init__( | |||
|
|
|||
| self.tasks = set() # type: Set[Task] | |||
| self.edges = set() # type: Set[Edge] | |||
| self.set_key_tasks([]) | |||
There was a problem hiding this comment.
I'd vote to include this as an init argument as well, so users don't have to declaratively call this method if they don't want to.
cicdw
approved these changes
Aug 7, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #26
Introduces a new method for flows called
set_key_tasks(). "Key tasks" are tasks that determine the state of the flow when it runs. By default, they are the flow's terminal tasks. However, the user may specify any other tasks in the flow.Operationally, this changes almost none of the flow run logic, except that it looks at the key tasks to determine state instead of the terminal tasks. The only change is that terminal tasks are still waited on (by the executor) and checked to make sure they're in a
Finishedstate. If - and only if - they have finished, the key tasks are checked to determine state. In other words: the flow must still be completely done before state is checked, even if the tasks that determine state are finished, themselves.