Add dask.blockwise.fuse_roots optimization#5451
Merged
mrocklin merged 8 commits intodask:masterfrom Nov 12, 2019
Merged
Conversation
This fuses data generation or access graphs into blockwise after converting everything to low level task graphs
eece3b6 to
b8a5d97
Compare
Member
Author
|
Tests pass. I'm going to wait until next week to make sure that the 2.7.0 release is smooth, and then I'll merge this in if there are no objections. |
Member
Author
|
@TomAugspurger @rabernat this is in. It should help avoid communication in situations where multiple arrays are loaded and then combined together as in |
Collaborator
|
Exciting! Thanks @mrocklin |
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fuses data generation or access graphs into blockwise after
converting everything to low level task graphs
cc @TomAugspurger we were just talking about this