FargateCluster hangs due to DescribeTaskDefinition requests

[marclipoff-curative]

What happened: When I simply call cluster = FargateCluster(), the process hangs. This is due to dask making a DescribeTaskDefinition request for every registered task definition. And there are a lot in the account

What you expected to happen: I expect there to be a way to avoid this

Minimal Complete Verifiable Example:

from dask_cloudprovider.aws import FargateCluster
cluster = FargateCluster()

Anything else we need to know?:

Environment:

• Dask version: 2022.4.0
• Python version: 3.9.7
• Operating System: mac
• Install method (conda, pip, source): pip

Cluster Dump State:

[marclipoff-curative]

@jacobtomlinson to you have a sense of when this will get picked up?

[jacobtomlinson]

I don't. Do you have any interest in raising a PR?

[marclipoff-curative]

I could certainly try. Browsing through the code, it's not immediately obvious to me where the DescribeTaskDefinition request is being made. Do you have any direction on that?

[marclipoff-curative]

I've solved this issue by using skip_cleanup=True.