Describe the issue:
Tasks are not uniformly distributed across all the available dask workers. The majority of the tasks, are getting piled up on a single worker. Is this an expected behavior?
Anything else we need to know?:
The cluster is adaptive and the rest of the workers were added after 1-2 minutes. Why the pending tasks are not getting routed to the new workers?
Environment:
- Dask version: 2.30.0
- Distributed version: 2.30.1
- Python version: 3.7.10
- Operating System: CentOS Linux 7 (Core)
- Install method (conda, pip, source): pip
Describe the issue:
Tasks are not uniformly distributed across all the available dask workers. The majority of the tasks, are getting piled up on a single worker. Is this an expected behavior?
Anything else we need to know?:
The cluster is adaptive and the rest of the workers were added after 1-2 minutes. Why the pending tasks are not getting routed to the new workers?
Environment: