Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove some uses of IOLoop._running #1512

Merged
merged 9 commits into from Nov 1, 2017
Merged

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Oct 31, 2017

Also some small cleanups.

One question revolves around the loop fixture. It asks the loop to stop at the end in case the test didn't stop already... and it turns out some tests don't seem to stop the loop indeed. Is it an error for a test not to do so?

@pitrou
Copy link
Member Author

pitrou commented Oct 31, 2017

@mrocklin

@mrocklin
Copy link
Member

and it turns out some tests don't seem to stop the loop indeed. Is it an error for a test not to do so?

Most tests will not stop the IOLoop. It is ok if they do or if they don't.

thread.start()
loop_started = threading.Event()
loop.add_callback(loop_started.set)
loop_started.wait()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pattern seems much cleaner than our other approaches. We might consider doing this in client.py as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think I can refactor a bit more code in that PR.

Otherwise, just mark this object "stopped".
"""
# XXX it would be more logical to stop an user-created loop
# if we started it, but that would break the current distributed API
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which situations would this break user code? Some breakage may be acceptable.

Copy link
Member Author

@pitrou pitrou Nov 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the problem is it currently breaks some of our tests. For example this one:

def test_Current(loop):
    with cluster() as (s, [a, b]):
        with Client(s['address'], loop=loop) as c:
            assert Client.current() is c
        with pytest.raises(ValueError):
            Client.current()
        with Client(s['address'], loop=loop) as c:
            assert Client.current() is c

As soon as the first with ends, the loop is closed, but it's passed again to the second Client instead, so the test hangs.

I don't know if there are use cases for continuing to use a loop after you closed the client that started it, but this is how it currently works.

(of course, that test doesn't need to re-use the same loop, but since it's written that way, it seems to hint that it's a valid idiom...)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is inconvenient then it might be reasonable to drop it. The next version will likely be a 0.x.0 bump, so we can break a few things.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not technically inconvenient, it just seems a bit inconsistent to have such assymetry in the start/stop responsabilities. But it probably wouldn't save any complexity to make it different.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think it's a good idea then go for it.

@pitrou
Copy link
Member Author

pitrou commented Nov 1, 2017

Will merge if no further comments.

@pitrou pitrou merged commit b63282c into dask:master Nov 1, 2017
@pitrou pitrou deleted the remove_some_running branch November 1, 2017 21:42
@pitrou
Copy link
Member Author

pitrou commented Nov 1, 2017

The one crucial remaining use of IOLoop._running is in the sync() function. I haven't found a reliable way to do without it.

@mrocklin
Copy link
Member

mrocklin commented Nov 1, 2017 via email

@mrocklin
Copy link
Member

mrocklin commented Nov 2, 2017

I think that we can just remove this functionality. I suspect that it does not come up in practice.

@pitrou
Copy link
Member Author

pitrou commented Nov 2, 2017

I think that we can just remove this functionality. I suspect that it does not come up in practice.

Actually, it comes up in the loop fixture, since we want to ensure the loop is stopped before closing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants