Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ThrottleExecutor can leak thread when not shutdown() #93

Closed
rohanpm opened this issue Jan 2, 2019 · 0 comments
Closed

ThrottleExecutor can leak thread when not shutdown() #93

rohanpm opened this issue Jan 2, 2019 · 0 comments

Comments

@rohanpm
Copy link
Owner

rohanpm commented Jan 2, 2019

Despite the issue #87 fix, which fixed most basic cases of thread leaks, ThrottleExecutor can still leak threads if it's not shut down.

It can be reproduced by adding a new test like this:

def test_no_leak_on_completed_futures(executor_ctor):
    no_extra_threads = partial(assert_no_extra_threads, thread_names())

    executor = executor_ctor()
    futures = [executor.submit(mult2, n) for n in [10, 20, 30]]
    results = [f.result() for f in futures]
    del executor
    del futures

    assert_soon(no_extra_threads)

...and running it with an Executors.with_poll(...).with_throttle(2).

The problem is that _submit_loop can still hold a reference to futures (and hence executor) via to_submit and job locals.

rohanpm added a commit that referenced this issue Jan 2, 2019
The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93
bors bot added a commit that referenced this issue Jan 2, 2019
95: Fix (another) thread leak in ThrottleExecutor r=rohanpm a=rohanpm

The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93

Co-authored-by: Rohan McGovern <rohan@mcgovern.id.au>
bors bot added a commit that referenced this issue Jan 2, 2019
95: Fix (another) thread leak in ThrottleExecutor r=rohanpm a=rohanpm

The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93

Co-authored-by: Rohan McGovern <rohan@mcgovern.id.au>
rohanpm added a commit that referenced this issue Jan 3, 2019
The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93
bors bot added a commit that referenced this issue Jan 3, 2019
95: Fix (another) thread leak in ThrottleExecutor r=rohanpm a=rohanpm

The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93

Co-authored-by: Rohan McGovern <rohan@mcgovern.id.au>
rohanpm added a commit that referenced this issue Jan 3, 2019
The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93
@rohanpm rohanpm closed this as completed Jan 3, 2019
bors bot added a commit that referenced this issue Jan 6, 2019
103: Fix (another) thread leak in ThrottleExecutor r=rohanpm a=rohanpm

The thread's main loop did "del executor" before waiting on the event,
but it could still hold a reference to the executor from 'to_submit'
or 'job'.

Fixes #93

Co-authored-by: Rohan McGovern <rohan@mcgovern.id.au>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant