Switch to cloud-provider-azure for cloud-controller-manager image

[feiskyer]

Because of kubernetes/kubernetes#81029, cloud-controller-manager image won't be released since v1.16.beta.x.

We should switch to hyperkube image when configuring cloud-controller-manager.

Update: I didn't notice there's a change also dropping cloud-controller-manager support in hyperkube kubernetes/kubernetes#81219. After this PR, we could only use the ccm image built from cloud-provider-azure repo.

Refer the example manifests here: https://github.com/kubernetes/cloud-provider-azure/tree/master/examples/out-of-tree.

[feiskyer]

@ritazh Could you help to take a look at this?

/assign @ritazh

[jackfrancis]

@feiskyer FYI updated the issue title

[feiskyer]

Just build and push images for Kubernetes v1.16:

mcr.microsoft.com/k8s/core/azure-cloud-controller-manager:v0.3.0
mcr.microsoft.com/k8s/core/azure-cloud-node-manager:v0.3.0

Their example manifests could be found at https://github.com/kubernetes/cloud-provider-azure/blob/master/examples/out-of-tree/cloud-node-manager.yaml and https://github.com/kubernetes/cloud-provider-azure/blob/master/examples/out-of-tree/cloud-controller-manager.yaml.

@jackfrancis could you help to the images as well as the new cloud-node-manager for v1.16.0 clusters?

[jackfrancis]

Thx @feiskyer will do!

[ritazh]

@jchauncey should we move to mcr.microsoft.com/oss/k8s now? It's going to be harder once the image is part of this code.

[mboersma]

I have a PR I'm about to put out there that assumes the URLs that @feiskyer mentioned above, and AKS Engine already has a constant in there MCRKubernetesImageBase = mcr.microsoft.com/k8s/core/. (This was added in #2050.)

Just sayin'–we can change these but as Rita said it becomes increasingly painful.

[jchauncey]

let me submit the pr to mcr to hook up our /oss path. once that is merged its easy to build the these 2 images and publish them to the new endpoint and just use that.

[ritazh]

Looks like we had some issues getting https://github.com/microsoft/mcr/pull/757 merged.

Until the new paths get sorted out, to unblock this PR and few other ones, maybe we can use the mcr.microsoft.com/k8s/core/ one for now then update with the new image path in a future PR? WDYT?

[feiskyer]

Until the new paths get sorted out, to unblock this PR and few other ones, maybe we can use the mcr.microsoft.com/k8s/core/ one for now then update with the new image path in a future PR? WDYT?

+1.

[jchauncey]

yeah sounds good to me