-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[DA] Update eager condition (#22746)
## Summary & Motivation Two things: ### update eager definition This subtly changes the definition of the `eager()` condition to deal with a particular annoying case. Essentially, imagine you have assets A -> B -> C, each with an eager policy, and starting off missing. On the first tick, all assets are detected as missing, and so a run is created targeting all of them. Now, this run progresses, and A is materialized. At this point in time, if we evaluate `B`, we will notice that `A` has now been newly updated, which would then result in `B` being materialized again (similar things would happen for `C`). In the AMP world, this is handled by fundamentally redefining the "parent updated" condition to ignore cases where the child will be materialized in the same run. While the same thing could be done here, I would argue that this pollutes the simplicity of the system and will result in a fair amount of confusion (there will be cases where people want the "pure" behavior, and the history page would now indicate that a parent _hadn't_ updated even though it did). The alternative path, taken here, is to avoid materializing an asset if there's already an in-progress run for it, and only materialize if there's been a parent update since the last time the asset was requested _or materialized_. This means that, while that initial run is progressing, the asset will not be re-materialized, and once that run completes, the asset will be in a "good state" again, as its parent has not updated more recently than the last time it was updated (or requested) ### removes / renames some properties This removes the "since_last_x" static constructors. We can always add them back in, but they were (in my opinion) a bit "too cute". They don't cover a high enough percentage of use cases to be worth having their own dedicated static methods, and just pollute the typeahead namespace. Also just renames "on_cron" to "cron" -- again a reversible decision but feels a bit cleaner. ## How I Tested These Changes
- Loading branch information
1 parent
939cb25
commit ef82147
Showing
5 changed files
with
28 additions
and
65 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters