garden plugins kubernetes cluster-init cannot find garden-docker-registry pod

[10ko]

Bug

Current Behavior

@mitchfriedman reports that occasionally when running garden plugins kubernetes cluster-init the script might fail wit hthe following error message:

Error: Could not find running pod for Deployment/garden-docker-registry
    at Object.<anonymous> (/snapshot/project/garden-service/tmp/dist/build/src/plugins/kubernetes/container/exec.js:0)
    at Generator.next (<anonymous>)
    at fulfilled (/snapshot/project/garden-service/tmp/dist/build/src/plugins/kubernetes/container/exec.js:0)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)

Expected behavior

The command should not fail

Reproducible example

It's not consistently happening but just running it multiple time would make the error happen.

Workaround

Running the command again usually will succeed.

Suggested solution(s)

It looks like this is happening because the way we check if a resource is ready in the waitForResources function.
It might happen that waitForResources returns before the garden-docker-registry pods are running, making the execInWorkload function after that fail.

[stale]

This issue has been automatically marked as stale because it hasn't had any activity in 60 days. It will be closed in 14 days if no further activity occurs (e.g. changing labels, comments, commits, etc.). Please feel free to tag a maintainer and ask them to remove the label if you think it doesn't apply. Thank you for submitting this issue and helping make Garden a better product!

[eysi09]

Is this still an issue?

[mitchfriedman]

I haven't seen it recently, but I also haven't been frequently initializing a new cluster.

[eysi09]

Ok. Perhaps we let stalebot do it's thing and re-open if we bump into this again.

[eysi09]

Bumping into this one again on our end.

[stale]

This issue has been automatically marked as stale because it hasn't had any activity in 60 days. It will be closed in 14 days if no further activity occurs (e.g. changing labels, comments, commits, etc.). Please feel free to tag a maintainer and ask them to remove the label if you think it doesn't apply. Thank you for submitting this issue and helping make Garden a better product!