Not revisiting task nodes and correctly incrementing parallelism #4318
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tracking issue
NA
Describe your changes
Currently, if the plugin state has already been observed we immediately return without incrementing
parallelism
or marking the task as "dirty", indicating that it has already been processed this round. This means that, in scenarios where a task is the downstream dependency of multiple nodes, tasks may be unnecessarily evaluated multiple times within a single round. More seriously, if the task state has no changed (ex. in same phase as previous round) then additional tasks will be scheduled to satisfy the parallelism. This means that, effectivelymax_parallelism
settings may be ignored.In this PR we correctly increment
parallelism
if the state has already been seen and mark the task node as "dirty" to indicate that it should not be evaluated again this round.Check all the applicable boxes
Screenshots
NA
Note to reviewers
NA