Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid hanging forever after a parallel job was killed #7834

Merged
merged 10 commits into from
Dec 12, 2022

Commits on Nov 23, 2022

  1. Replace multiprocessing.pool with concurrent.futures.ProcessPoolExecu…

    …tor to avoid deadlocks.
    
    In a multiprocessing.pool, if a process terminates in a non-clean fashion
    (for example, due to OOM or a segmentation fault), the pool will silently
    replace said process, but the work that the process was supposed to do
    will never be done, causing pylint to hang indefinitely.
    The concurrent.futures.ProcessPoolExecutor will raise a
    BrokenProcessPool exception in that case, avoiding the hang.
    daniel-wer committed Nov 23, 2022
    Configuration menu
    Copy the full SHA
    fc1baf7 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2022

  1. Configuration menu
    Copy the full SHA
    d5e17d8 View commit details
    Browse the repository at this point in the history
  2. add whatsnew fragment

    daniel-wer committed Nov 24, 2022
    Configuration menu
    Copy the full SHA
    2657772 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2022

  1. Add a test that an error in the initializer for the parallel jobs, do…

    …esn't
    
    lead to a deadlock.
    daniel-wer committed Nov 28, 2022
    Configuration menu
    Copy the full SHA
    2e47049 View commit details
    Browse the repository at this point in the history

Commits on Dec 5, 2022

  1. Configuration menu
    Copy the full SHA
    236e5e9 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2022

  1. Configuration menu
    Copy the full SHA
    61b8342 View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2022

  1. Check availability of concurrent.futures module instead of multiproce…

    …ssing for single process fallback.
    daniel-wer committed Dec 8, 2022
    Configuration menu
    Copy the full SHA
    a52cb6c View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2022

  1. Small fixes

    DanielNoord authored Dec 10, 2022
    Configuration menu
    Copy the full SHA
    30af13a View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2022

  1. Update tests/test_check_parallel.py

    Co-authored-by: Daniël van Noord <13665637+DanielNoord@users.noreply.github.com>
    daniel-wer and DanielNoord authored Dec 12, 2022
    Configuration menu
    Copy the full SHA
    e164bea View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    69729db View commit details
    Browse the repository at this point in the history