Ensure workers do not kill on restart

[fjetter]

Closes #7312
Supersedes #7323

closes #7321

xref #7320

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    29 files  +    10      29 suites  +10   12h 10m 14s ⏱️ + 5h 47m 38s
 4 084 tests +     1   3 957 ✅  -      9    112 💤  -   2  15 ❌ +12 
55 244 runs  +21 365  52 775 ✅ +20 398  2 431 💤 +932  38 ❌ +35 

For more details on these failures, see this check.

Results for commit 014ed07. ± Comparison against base commit eab58be.

This pull request removes 7 and adds 8 tests. Note that renamed tests count towards both.

distributed.diagnostics.tests.test_memory_sampler ‑ test_async
distributed.tests.test_client ‑ test_restart_workers_kill_timeout[False]
distributed.tests.test_client ‑ test_restart_workers_kill_timeout[True]
distributed.tests.test_failed_workers ‑ test_worker_ttl_restarts_worker[False]
distributed.tests.test_failed_workers ‑ test_worker_ttl_restarts_worker[True]
distributed.tests.test_scheduler ‑ test_restart_nanny_timeout_exceeded
distributed.tests.test_scheduler ‑ test_restart_worker_rejoins_after_timeout_expired

distributed.deploy.tests.test_local ‑ test_localcluster_restart
distributed.tests.test_failed_workers ‑ test_worker_ttl_restarts_worker[None]
distributed.tests.test_failed_workers ‑ test_worker_ttl_restarts_worker[event_loop]
distributed.tests.test_failed_workers ‑ test_worker_ttl_restarts_worker[threadpool]
distributed.tests.test_nanny ‑ test_restart_stress[kill]
distributed.tests.test_nanny ‑ test_restart_stress[restart]
distributed.tests.test_nanny ‑ test_worker_start_exception_after_restart[kill]
distributed.tests.test_nanny ‑ test_worker_start_exception_after_restart[restart]

♻️ This comment has been updated with latest results.

[hendrikmakait]

What's the status of this PR? Do you need help resolving conflicts?