[Docs] Add Section on Expectations to Graph Page#5764
Conversation
* Resolves dask/distributed#3350 * Adds a section in the documentation that discusses how Futures are handled when tasks operate on them `in-place`
mrocklin
left a comment
mrocklin
left a comment
There was a problem hiding this comment.
Thanks @devin-petersohn ! I left a few bits of micro-feedback. I was a little more verbose than usual, just because I'm hoping to see you around these parts again :) Hope that's alright.
Co-Authored-By: Matthew Rocklin <mrocklin@gmail.com>
mrocklin
left a comment
mrocklin
left a comment
There was a problem hiding this comment.
Thanks @devin-petersohn ! A few more small comments. It looks like we're converging though.
docs/source/graphs.rst
Outdated
| When a task is submitted to Dask for execution, there are a number of assumptions | ||
| that are made about that task. In general, tasks with side-effects that alter the | ||
| state of a future in-place are not recommended. Values stored by Dask are mutable, | ||
| and can be updated in-place. For example, consider a workflow involving a |
There was a problem hiding this comment.
On second thought the sentence
Values stored by dask are mutable and can be updated in place
might be interpretted as
it is permissible to update data in place
raher than
nothing stops you from shooting yourself in the foot
Maybe instead of this sentence we say "Modifying data stored in Dask can have unintended consequences".
There was a problem hiding this comment.
Yes, I agree. I think there are probably cases where the behavior is non-deterministic or introduces race conditions, so maybe we should add that it's not officially a supported functionality, or explicitly mention the race condition possibility.
Co-Authored-By: Matthew Rocklin <mrocklin@gmail.com>
|
Thanks @mrocklin, I'm happy to keep updating as necessary. |
|
Merging this in. Thanks @devin-petersohn ! |
2 participants
in-placecc @mrocklin
black dask/flake8 dask