-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TASK LOOPING (in both Core and Cloud) #1356
Conversation
@@ -467,6 +476,32 @@ class Finished(State): | |||
color = "#003ccb" | |||
|
|||
|
|||
class Looped(Finished): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes me wonder if both Looped
and Mapped
should become Looping
and Mapping
since they imply that more work is going to be done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea I thought about this; I'm happy to change it but here's my reasoning for Looped
: when a Task enters a Looped
state it lets us know that a single iteration of the Loop was complete. The full "looping" mechanism actually involves Looped -> Running -> Looped -> Running
.
This is in contrast to a Running
state - when a Task is in a Running
state, it actually means there is work currently being done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm in
Thanks for contributing to Prefect!
Please describe your work and make sure your PR:
CHANGELOG.md
(if appropriate)docs/outline.toml
for API reference docs (if appropriate)Note that your PR will not be reviewed unless all three boxes are checked.
What does this PR change?
This PR introduces "Task Looping" (first described in PIN 11). The implementation here will actually work in Cloud almost immediately (once the Cloud serializers are updated).
In order to make Looping work off-the-shelf, this PR:
Looped
state which carries theloop_count
LOOP
signal for triggering a loopcached_inputs
mechanism for conveying relevant data to retriestask_loop_count
in contextWhy is this PR important?
This PR furthers the ability for Flows to dynamically spawn tasks. Mapping is to for loops as Looping is to while loops.
This of course will need extensive documentation to fully expose.