Do not rely on logging for SubprocessCluster
#8398
Conversation
hendrikmakait
requested review from
jacobtomlinson and
fjetter
as code owners
Unit Test ResultsSee test report for an extended history of previous test failures. This is useful for diagnosing flaky tests. 27 files ± 0 27 suites ±0 11h 52m 36s ⏱️ + 14m 41s For more details on these failures, see this check. Results for commit 2c49f6b. ± Comparison against base commit c408b6a. |
|
The new test is very flaky: |
| with new_config_file( | ||
| {"distributed": {"logging": {"distributed": logging.CRITICAL + 1}}} | ||
| ): | ||
| await asyncio.wait_for(_start(), timeout=2) |
There was a problem hiding this comment.
I don't understand what's the purpose of this super-short timeout, and why the timeout integrated in gen_test is not what we want in this case?
There was a problem hiding this comment.
I wrote the test before the fix and didn't want to wait the entire 30 seconds while fixing.
|
Attempt fix on #8413 |
Partially addresses #8392, #8393
This PR relies on the
scheduler_fileto propagate the scheduler address to the worker.Note:
Conceptually, the same fix can be applied to the
SSHClusterbut things get mildly more complicated because one might have to deal with different file systems for the client and the scheduler.pre-commit run --all-files