[WIP] Trivial deps ignored account already transferring

[fjetter]

Closes #8177

This changes how workers are picked for task placement. Currently, workers with a dependency are very strongly favored. In fact, they are the only viable candidates regardless of size of the dependencies or other factors like occupancy.

This is an attempt to rectify this. These changes are actually already a couple months old. Back then I had to drop this investigation due to inconclusive benchmark results. I'm currently rerunning them to see where we are. This is obviously a very sensitive change we have to be careful about

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       27 files  ±  0         27 suites  ±0   13h 27m 39s ⏱️ + 10m 29s
  3 929 tests +  3    3 796 ✔️  - 11     121 💤 +  4  12 ❌ +10 
49 430 runs  +57  46 934 ✔️  - 19  2 457 💤 +40  39 ❌ +36 

For more details on these failures, see this check.

Results for commit dba7271. ± Comparison against base commit e98dcb1.

This pull request skips 4 tests.

distributed.tests.test_scheduler ‑ test_decide_worker_coschedule_order_neighbors[nthreads0-0]
distributed.tests.test_scheduler ‑ test_decide_worker_coschedule_order_neighbors[nthreads0-1]
distributed.tests.test_scheduler ‑ test_decide_worker_coschedule_order_neighbors[nthreads1-0]
distributed.tests.test_scheduler ‑ test_decide_worker_coschedule_order_neighbors[nthreads1-1]

♻️ This comment has been updated with latest results.