kube-dns-v20 deployment contains wrong docker image properties

[mpluhar]

What happened:

kube-dns-v20 deployment contains wrong image properties.
AKS must have rolled out an update of the kube-dns-v20 deployment adding a sidecar. The image name for the sidecar container is wrong

What you expected to happen:

Deployment should contain the correct information pointing to an existing image

How to reproduce it (as minimally and precisely as possible):

The cluster is in Australia East
kubectl -n kube-system get deployment kube-dns-v20 -o yaml | grep image:

    image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13
    image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10
    image: k8s.gcr.io/exechealthz-amd64:1.2
    **image: k8s.gcr.io/**

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.2", GitCommit:"17c77c7898218073f14c8d573582e8d2313dc740", GitTreeState:"clean", BuildDate:"2018-10-24T06:54:59Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.3", GitCommit:"a4529464e4629c21224b3d52edfe0ea91b072862", GitTreeState:"clean", BuildDate:"2018-09-09T17:53:03Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
mpluhar@sputnik:~/clearpoint/tower/src/tcd-temp$

• Size of cluster (how many worker nodes are in the cluster?)

5

• General description of workloads in the cluster (e.g. HTTP microservices, Java app, Ruby on Rails, machine learning, etc.)

Microservices, .NET

• Others:

[strtdusty]

I have opened a support case for this as well.

[mpluhar]

@strtdusty We rebuilt a cluster this morning and the deployment for the DNS is fine now. Not sure if that is an option for you?

[strtdusty]

Nope, rebuilding clusters aren't an option anymore :)

[mpluhar]

Output of "kubectl -n kube-system get deployments kube-dns-v20 -o yaml | grep image:"

    image: k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.13
    image: k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10
    image: k8s.gcr.io/exechealthz-amd64:1.2
    image: k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10

I think you can edit the deployment for kube-dns-v20, but AKS might override it.

[patrickmslatteryvt]

We experienced this yesterday too, I was able to fix it this morning by deleting the kube-dns pods with the InvalidImageName status and it redeployed new pods with the correct images.

[motarski]

Any progress on this. I tried to edit the deployment but it gets overriden after first pod gets created?

[mpluhar]

Any progress on this. I tried to edit the deployment but it gets overriden after first pod gets created?

Did you try to delete the pods?

[jnoller]

Closing this issue as old/stale.

If this issue still comes up, please confirm you are running the latest AKS release. If you are on the latest release and the issue can be re-created outside of your specific cluster please open a new github issue.

If you are only seeing this behavior on clusters with a unique configuration (such as custom DNS/VNet/etc) please open an Azure technical support ticket.