-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change to dask.order
: be more eager at times
#7929
Merged
Merged
Commits on Jul 23, 2021
-
Small change to
dask.order
. Be more eager at times.`dask.order` will now more eagerly compute dependents if the parent is expected to be released soon. The simplified explanation is that if running a single dependent will allow the parent to be released from memory, then we should do it. Although this is a wash--because we computed one thing and released another, so net zero--this can have big implications. Computing more eagerly earlier can give more opportunities for releasing memory (such as computing one thing and releasing two). The more complicated (and correct) explanation is that we compute a dependent if doing so will let the parent be released *soon*. Computing the depenend *may* allow the parent to be released immediately, but not necessarily so. But, there is the expectation that the parent will be able to be released *soon*, because all of its other dependents are already in one of the inner stacks. Hence, this can introduce local sub-optimal behavior (more work and more memory) with the goal of being more globally optimal. I think this is pretty cool!
Configuration menu - View commit details
-
Copy full SHA for c4e5eac - Browse repository at this point
Copy the full SHA c4e5eacView commit details
Commits on Jul 24, 2021
-
Add test and descriptions. Clean up current approach.
Give up trying to add this same trick to the "slow path". Doing so is tricky.
Configuration menu - View commit details
-
Copy full SHA for c3aef63 - Browse repository at this point
Copy the full SHA c3aef63View commit details
Commits on Jul 25, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 09a83cf - Browse repository at this point
Copy the full SHA 09a83cfView commit details
Commits on Aug 2, 2021
-
Update
dask.order
to be smarter about computing single chains when ……available. This is much like the previous version, but better (and more complicated). Unlike before, this is now not over-eager. It will only eagerly compute a node when it is known that doing so will free a node.
Configuration menu - View commit details
-
Copy full SHA for 1f104ea - Browse repository at this point
Copy the full SHA 1f104eaView commit details -
Configuration menu - View commit details
-
Copy full SHA for f0e7368 - Browse repository at this point
Copy the full SHA f0e7368View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1362687 - Browse repository at this point
Copy the full SHA 1362687View commit details
Commits on Aug 3, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 2ebde8d - Browse repository at this point
Copy the full SHA 2ebde8dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 893f43f - Browse repository at this point
Copy the full SHA 893f43fView commit details -
Configuration menu - View commit details
-
Copy full SHA for bda2d2b - Browse repository at this point
Copy the full SHA bda2d2bView commit details
Commits on Aug 4, 2021
-
Configuration menu - View commit details
-
Copy full SHA for 1d8ef1a - Browse repository at this point
Copy the full SHA 1d8ef1aView commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.