New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubeadm: sort pod Volumes and VolumeMounts #70027

Merged
merged 1 commit into from Oct 23, 2018

Conversation

@bart0sh
Contributor

bart0sh commented Oct 19, 2018

What type of PR is this?
/kind bug

What this PR does / why we need it:
Order of Volumes and VolumeMounts in the pod objects created by
kubeadm is undefined as they're represended as maps in the
controlPlaneHostPathMounts struct.

This influences 'kubeadm upgrade' logic in a way that even when
manifest of the component is not changed kubeadm tries to upgrade
it because most of the time current and new pods are not equal
due to the different order of Volumes and VolumeMounts.

For example 'kubeadm apply diff' almost always shows difference
in Volumes and VolumeMounts because of this:

     volumeMounts:
+    - mountPath: /etc/kubernetes/pki
+      name: k8s-certs
+      readOnly: true
     - mountPath: /etc/ssl/certs
       name: ca-certs
+      readOnly: true
+    - mountPath: /etc/pki
+      name: etc-pki
+      readOnly: true
+    - mountPath: /usr/share/ca-certificates
+      name: usr-share-ca-certificates
       readOnly: true
     - mountPath: /etc/ca-certificates
       name: etc-ca-certificates
       readOnly: true
-    - mountPath: /etc/pki
-      name: etc-pki
-      readOnly: true
-    - mountPath: /etc/kubernetes/pki
-      name: k8s-certs
-      readOnly: true
-    - mountPath: /usr/share/ca-certificates
-      name: usr-share-ca-certificates
-      readOnly: true

Sorting Volumes and VolumeMounts should fix this issue and help
to avoid unnecessary upgrades.

Which issue(s) this PR fixes:
Fixes kubernetes/kubeadm#1054

Does this PR introduce a user-facing change?:

kubeadm: fix unnecessary upgrades caused by undefined order of Volumes and VolumeMounts in manifests
kubeadm: sort pod Volumes and VolumeMounts
Order of Volumes and VolumeMounts in the pod objects created by
kubeadm is undefined as they're represended as maps in the
controlPlaneHostPathMounts struct.

This influences 'kubeadm upgrade' logic in a way that even when
manifest of the component is not changed kubeadm tries to upgrade
it because most of the time current and new pods are not equal
due to the different order of Volumes and VolumeMounts.

For example 'kubeadm apply diff' almost always shows difference
in Volumes and VolumeMounts because of this:
     volumeMounts:
+    - mountPath: /etc/kubernetes/pki
+      name: k8s-certs
+      readOnly: true
     - mountPath: /etc/ssl/certs
       name: ca-certs
+      readOnly: true
+    - mountPath: /etc/pki
+      name: etc-pki
+      readOnly: true
+    - mountPath: /usr/share/ca-certificates
+      name: usr-share-ca-certificates
       readOnly: true
     - mountPath: /etc/ca-certificates
       name: etc-ca-certificates
       readOnly: true
-    - mountPath: /etc/pki
-      name: etc-pki
-      readOnly: true
-    - mountPath: /etc/kubernetes/pki
-      name: k8s-certs
-      readOnly: true
-    - mountPath: /usr/share/ca-certificates
-      name: usr-share-ca-certificates
-      readOnly: true

Sorting Volumes and VolumeMounts should fix this issue and help
to avoid unnecessary upgrades.
@bart0sh

This comment has been minimized.

Contributor

bart0sh commented Oct 19, 2018

@k8s-ci-robot k8s-ci-robot requested a review from neolit123 Oct 19, 2018

@neolit123

@bart0sh thank you for this PR.

i think the PR is pretty self-explanatory and the sorted volumes is something we should definitely have.

one small request: please prefix the kubeadm release notes like so - e.g. kubeadm: note goes here.....
thanks

/lgtm
/assign @timothysc
/assign @liztio
(as per upgrade diff).

@timothysc

/lgtm
/approve

@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Oct 23, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bart0sh, timothysc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 46ec13c into kubernetes:master Oct 23, 2018

18 checks passed

cla/linuxfoundation bart0sh authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment