Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timeout and check on write #653

Merged
merged 2 commits into from
Nov 11, 2016
Merged

Timeout and check on write #653

merged 2 commits into from
Nov 11, 2016

Conversation

mrocklin
Copy link
Member

@mrocklin mrocklin commented Nov 11, 2016

The core.write coroutine waits on IOStream.write futures. However, there are
some cases where IOStream.write futures don't complete. Notably if something
else writes to the same stream before we're done waiting then the first future
is forgotten and hangs forever.

If `write` is called again before that `.Future` has resolved, the
previous future will be orphaned and will never resolve.

This solution wraps the wait in a timeout and checks the stream's
write_buffer as a secondary check for completion. There are probably better
solutions that don't rely on the timeout but this should work fine.

cc @pitrou

The core.write coroutine waits on IOStream.write futures.  However, there are
some cases where IOStream.write futures don't complete.  Notably if something
else writes to the same stream before we're done waiting then the first future
is forgotten and hangs forever.

    If `write` is called again before that `.Future` has resolved, the
    previous future will be orphaned and will never resolve.

This solution wraps the wait in a timeout and checks the stream's
write_buffer as a secondary check for completion.  There are probably better
solutions but this seems to work fine.
This was recently moved to core.write
@mrocklin mrocklin merged commit dbb5c79 into dask:master Nov 11, 2016
@mrocklin mrocklin deleted the timeout-on-write branch November 11, 2016 15:32
@mrocklin mrocklin mentioned this pull request Nov 12, 2016
pitrou added a commit to pitrou/distributed that referenced this pull request Nov 14, 2016
Does away with the timeout and looking up a private attribute on IOStream.
Refs PR dask#653.
pitrou added a commit to pitrou/distributed that referenced this pull request Nov 14, 2016
Does away with the timeout and looking up a private attribute on IOStream.
Refs PR dask#653.
pitrou added a commit to pitrou/distributed that referenced this pull request Nov 14, 2016
Does away with the timeout and looking up a private attribute on IOStream.
Refs PR dask#653.
@pitrou pitrou mentioned this pull request Nov 14, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant