-
Notifications
You must be signed in to change notification settings - Fork 254
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[epic] Kubernetes 1.22 Charts Validation and Image Updates #1892
Comments
Why do you want to |
If we don't end up needing to update them then feel free to mark those done without any change; just add a note as I did for the |
Should I update images, charts, etc on rke2 for calico, multus, coredns, etc? Or we will do a big update of all components in one PR? |
If you can link to the rke2-charts updates I can pull those and the image list updates in to my 1.22 PR before I merge it. I've already got the canal and metrics-server chart updates in there. |
For calico, this is the chart version:
And the images:
The apiserver is not deployed because they forgot to add it in the operator v1.20.3, so either we remove it and add it when we update the operator or we must remove it from the CI test as we do with calico-ctl For canal, you said you had already the one that fixes the deprecated API. We are missing a new rke2-chart version of coredns, multus and canal with the build-base update. I can take care of that tomorrow morning |
I was missing:
|
Multus:
|
Confirmed core functionality (canal, coredns, metrics-server, ingress-nginx): [root@centos01 ~]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cloud-controller-manager-centos01.lan.khaus 1/1 Running 0 40s
kube-system etcd-centos01.lan.khaus 1/1 Running 0 77s
kube-system helm-install-rke2-canal--1-wmxf9 0/1 Completed 0 78s
kube-system helm-install-rke2-coredns--1-vdrr2 0/1 Completed 0 78s
kube-system helm-install-rke2-ingress-nginx--1-z2jfn 0/1 Completed 0 78s
kube-system helm-install-rke2-metrics-server--1-rrhw9 0/1 Completed 0 78s
kube-system kube-apiserver-centos01.lan.khaus 1/1 Running 0 64s
kube-system kube-controller-manager-centos01.lan.khaus 1/1 Running 0 53s
kube-system kube-proxy-centos01.lan.khaus 1/1 Running 0 50s
kube-system kube-scheduler-centos01.lan.khaus 1/1 Running 0 50s
kube-system rke2-canal-kw4mf 1/2 Running 0 69s
kube-system rke2-coredns-rke2-coredns-54498b9b86-jbc4l 1/1 Running 0 69s
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-n6k77 1/1 Running 0 69s
kube-system rke2-ingress-nginx-controller-6rzhn 1/1 Running 0 36s
kube-system rke2-metrics-server-5b745c5ccb-jkfr5 1/1 Running 0 42s
[root@centos01 ~]# crictl image ls
IMAGE TAG IMAGE ID SIZE
docker.io/rancher/hardened-calico v3.20.1-build20211006 1444b9c6f8ebf 183MB
docker.io/rancher/hardened-cluster-autoscaler v1.8.5-build20211006 1b8d2fba888e1 40.8MB
docker.io/rancher/hardened-coredns v1.8.5-build20211006 6e03a70a0d90b 48MB
docker.io/rancher/hardened-etcd v3.5.0-k3s2-build20210915 2e6c3409c04ce 49MB
docker.io/rancher/hardened-flannel v0.14.0-build20211006 80753b7faeaa5 95.7MB
docker.io/rancher/hardened-k8s-metrics-server v0.5.0-build20210915 d94f6027f5853 47MB
docker.io/rancher/hardened-kubernetes v1.22.2-rke2r1-build20210924 ea6128c5fdee3 221MB
docker.io/rancher/klipper-helm v0.6.5-build20210915 710ba7ee5997e 84.5MB
docker.io/rancher/mirrored-ingress-nginx-kube-webhook-certgen v1.0 17e55ec30f203 18.6MB
docker.io/rancher/nginx-ingress-controller nginx-1.0.2-hardened1 969b4c43d74b1 228MB
docker.io/rancher/pause 3.5 69bfc1b271447 299kB
docker.io/rancher/rke2-cloud-provider v0.0.2-build20210915 3c506666fbd5a 46.8MB
docker.io/rancher/rke2-runtime v1.22.2-dev-6f09e7cb e3fb668cf9fd4 307MB
[root@centos01 ~]# kubectl top pods -A
NAMESPACE NAME CPU(cores) MEMORY(bytes)
kube-system cloud-controller-manager-centos01.lan.khaus 3m 25Mi
kube-system etcd-centos01.lan.khaus 14m 47Mi
kube-system kube-apiserver-centos01.lan.khaus 76m 342Mi
kube-system kube-controller-manager-centos01.lan.khaus 12m 78Mi
kube-system kube-proxy-centos01.lan.khaus 1m 20Mi
kube-system kube-scheduler-centos01.lan.khaus 3m 30Mi
kube-system rke2-canal-kw4mf 24m 92Mi
kube-system rke2-coredns-rke2-coredns-54498b9b86-jbc4l 2m 23Mi
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-n6k77 12m 19Mi
kube-system rke2-ingress-nginx-controller-6rzhn 2m 89Mi
kube-system rke2-metrics-server-5b745c5ccb-jkfr5 3m 26Mi The calico-node pod has errors though:
|
Confirmed working with Calico CNI: [root@centos01 ~]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
calico-system calico-kube-controllers-7696dd4d98-jjzkr 1/1 Running 0 2m3s
calico-system calico-node-w6j4n 1/1 Running 0 2m3s
calico-system calico-typha-5888bd77b5-4sndx 1/1 Running 0 2m3s
kube-system cloud-controller-manager-centos01.lan.khaus 1/1 Running 0 114s
kube-system etcd-centos01.lan.khaus 1/1 Running 0 2m35s
kube-system helm-install-rke2-calico--1-477zh 0/1 Completed 2 2m32s
kube-system helm-install-rke2-calico-crd--1-dcxjq 0/1 Completed 0 2m32s
kube-system helm-install-rke2-coredns--1-4spzj 0/1 Completed 0 2m32s
kube-system helm-install-rke2-ingress-nginx--1-tt9x2 0/1 Completed 0 2m32s
kube-system helm-install-rke2-metrics-server--1-s2rbq 0/1 Completed 0 2m32s
kube-system kube-apiserver-centos01.lan.khaus 1/1 Running 0 2m14s
kube-system kube-controller-manager-centos01.lan.khaus 1/1 Running 0 2m7s
kube-system kube-proxy-centos01.lan.khaus 1/1 Running 0 2m2s
kube-system kube-scheduler-centos01.lan.khaus 1/1 Running 0 2m3s
kube-system rke2-coredns-rke2-coredns-54498b9b86-shgq7 1/1 Running 0 2m21s
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-4lpcw 1/1 Running 0 2m21s
kube-system rke2-ingress-nginx-controller-pl9kh 1/1 Running 0 94s
kube-system rke2-metrics-server-5b745c5ccb-hqdv2 1/1 Running 0 105s
tigera-operator tigera-operator-99686b585-x76x8 1/1 Running 0 2m8s
[root@centos01 ~]# crictl image ls
IMAGE TAG IMAGE ID SIZE
docker.io/rancher/hardened-cluster-autoscaler v1.8.5-build20211006 1b8d2fba888e1 40.8MB
docker.io/rancher/hardened-coredns v1.8.5-build20211006 6e03a70a0d90b 48MB
docker.io/rancher/hardened-etcd v3.5.0-k3s2-build20210915 2e6c3409c04ce 49MB
docker.io/rancher/hardened-k8s-metrics-server v0.5.0-build20210915 d94f6027f5853 47MB
docker.io/rancher/hardened-kubernetes v1.22.2-rke2r1-build20210924 ea6128c5fdee3 221MB
docker.io/rancher/klipper-helm v0.6.5-build20210915 710ba7ee5997e 84.5MB
docker.io/rancher/mirrored-calico-cni v3.20.1 e69ccb66d1b65 48.4MB
docker.io/rancher/mirrored-calico-kube-controllers v3.20.1 5df320a38f63a 26.1MB
docker.io/rancher/mirrored-calico-node v3.20.1 355c1ee44040b 57.7MB
docker.io/rancher/mirrored-calico-operator v1.20.3 98e04bee27575 14.1MB
docker.io/rancher/mirrored-calico-pod2daemon-flexvol v3.20.1 55fa5eb71e097 9.36MB
docker.io/rancher/mirrored-calico-typha v3.20.1 8473ae43d01b8 24.2MB
docker.io/rancher/mirrored-ingress-nginx-kube-webhook-certgen v1.0 17e55ec30f203 18.6MB
docker.io/rancher/nginx-ingress-controller nginx-1.0.2-hardened1 969b4c43d74b1 228MB
docker.io/rancher/pause 3.5 69bfc1b271447 299kB
docker.io/rancher/rke2-cloud-provider v0.0.2-build20210915 3c506666fbd5a 46.8MB
docker.io/rancher/rke2-runtime v1.22.2-dev-6f09e7cb e3fb668cf9fd4 307MB
[root@centos01 ~]# kubectl top pods -A
NAMESPACE NAME CPU(cores) MEMORY(bytes)
calico-system calico-kube-controllers-7696dd4d98-jjzkr 10m 17Mi
calico-system calico-typha-5888bd77b5-4sndx 5m 25Mi
kube-system cloud-controller-manager-centos01.lan.khaus 7m 16Mi
kube-system etcd-centos01.lan.khaus 33m 50Mi
kube-system kube-apiserver-centos01.lan.khaus 125m 446Mi
kube-system kube-controller-manager-centos01.lan.khaus 31m 70Mi
kube-system kube-proxy-centos01.lan.khaus 6m 15Mi
kube-system kube-scheduler-centos01.lan.khaus 9m 28Mi
kube-system rke2-coredns-rke2-coredns-54498b9b86-shgq7 3m 11Mi
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-4lpcw 20m 17Mi
kube-system rke2-ingress-nginx-controller-pl9kh 21m 93Mi
kube-system rke2-metrics-server-5b745c5ccb-hqdv2 16m 22Mi
tigera-operator tigera-operator-99686b585-x76x8 9m 25Mi |
Confirmed working with Cilium CNI: [root@centos01 ~]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cilium-ltlzv 1/1 Running 0 6m53s
kube-system cilium-node-init-tkfg6 1/1 Running 0 6m53s
kube-system cilium-operator-85f67b5cb7-g94zw 0/1 Pending 0 6m53s
kube-system cilium-operator-85f67b5cb7-lkpsp 1/1 Running 0 6m53s
kube-system cloud-controller-manager-centos01.lan.khaus 1/1 Running 0 7m8s
kube-system etcd-centos01.lan.khaus 1/1 Running 0 7m4s
kube-system helm-install-rke2-cilium--1-qqtp8 0/1 Completed 0 7m1s
kube-system helm-install-rke2-coredns--1-j8rm5 0/1 Completed 0 7m1s
kube-system helm-install-rke2-ingress-nginx--1-s76bl 0/1 Completed 0 7m1s
kube-system helm-install-rke2-metrics-server--1-9hx5v 0/1 Completed 0 7m1s
kube-system kube-apiserver-centos01.lan.khaus 1/1 Running 0 6m41s
kube-system kube-controller-manager-centos01.lan.khaus 1/1 Running 0 6m32s
kube-system kube-proxy-centos01.lan.khaus 1/1 Running 0 7m9s
kube-system kube-scheduler-centos01.lan.khaus 1/1 Running 0 6m40s
kube-system rke2-coredns-rke2-coredns-54498b9b86-qtk9g 1/1 Running 0 6m53s
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-qw5wx 1/1 Running 3 (5s ago) 6m53s
kube-system rke2-ingress-nginx-controller-sgsln 1/1 Running 0 6m28s
kube-system rke2-metrics-server-5b745c5ccb-nwmsb 1/1 Running 0 6m33s
[root@centos01 ~]# crictl image ls
IMAGE TAG IMAGE ID SIZE
docker.io/rancher/hardened-cluster-autoscaler v1.8.5-build20211006 1b8d2fba888e1 40.8MB
docker.io/rancher/hardened-coredns v1.8.5-build20211006 6e03a70a0d90b 48MB
docker.io/rancher/hardened-etcd v3.5.0-k3s2-build20210915 2e6c3409c04ce 49MB
docker.io/rancher/hardened-k8s-metrics-server v0.5.0-build20210915 d94f6027f5853 47MB
docker.io/rancher/hardened-kubernetes v1.22.2-rke2r1-build20210924 ea6128c5fdee3 221MB
docker.io/rancher/klipper-helm v0.6.5-build20210915 710ba7ee5997e 84.5MB
docker.io/rancher/mirrored-cilium-cilium v1.10.4 f41c862f6e85c 150MB
docker.io/rancher/mirrored-cilium-operator-generic v1.10.4 c0ea117ea1bd7 16MB
docker.io/rancher/mirrored-cilium-startup-script 62bfbe88c17778aad7bef9fa57ff9e2d4a9ba0d8 eca17eaf44ca4 5.25MB
docker.io/rancher/mirrored-ingress-nginx-kube-webhook-certgen v1.0 17e55ec30f203 18.6MB
docker.io/rancher/nginx-ingress-controller nginx-1.0.2-hardened1 969b4c43d74b1 228MB
docker.io/rancher/pause 3.5 69bfc1b271447 299kB
docker.io/rancher/rke2-cloud-provider v0.0.2-build20210915 3c506666fbd5a 46.8MB
docker.io/rancher/rke2-runtime v1.22.2-dev-6f09e7cb e3fb668cf9fd4 307MB
[root@centos01 ~]# kubectl top pods -A
NAMESPACE NAME CPU(cores) MEMORY(bytes)
kube-system cilium-ltlzv 4m 89Mi
kube-system cilium-node-init-tkfg6 1m 2Mi
kube-system cilium-operator-85f67b5cb7-lkpsp 1m 22Mi
kube-system cloud-controller-manager-centos01.lan.khaus 3m 21Mi
kube-system etcd-centos01.lan.khaus 16m 48Mi
kube-system kube-apiserver-centos01.lan.khaus 59m 373Mi
kube-system kube-controller-manager-centos01.lan.khaus 11m 59Mi
kube-system kube-proxy-centos01.lan.khaus 1m 14Mi
kube-system kube-scheduler-centos01.lan.khaus 3m 24Mi
kube-system rke2-coredns-rke2-coredns-54498b9b86-qtk9g 2m 16Mi
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-qw5wx 19m 12Mi
kube-system rke2-ingress-nginx-controller-sgsln 2m 89Mi
kube-system rke2-metrics-server-5b745c5ccb-nwmsb 3m 22Mi |
Confirmed working with Multus+Canal: [root@centos01 ~]# kubectl get pods -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cloud-controller-manager-centos01.lan.khaus 1/1 Running 0 105s
kube-system etcd-centos01.lan.khaus 1/1 Running 0 101s
kube-system helm-install-rke2-canal--1-p5ll7 0/1 Completed 0 96s
kube-system helm-install-rke2-coredns--1-svh7m 0/1 Completed 0 96s
kube-system helm-install-rke2-ingress-nginx--1-b24mx 0/1 Completed 0 96s
kube-system helm-install-rke2-metrics-server--1-wr5l6 0/1 Completed 0 96s
kube-system helm-install-rke2-multus--1-m8pkg 0/1 Completed 0 96s
kube-system kube-apiserver-centos01.lan.khaus 1/1 Running 0 81s
kube-system kube-controller-manager-centos01.lan.khaus 1/1 Running 0 72s
kube-system kube-multus-ds-px2fl 1/1 Running 0 88s
kube-system kube-proxy-centos01.lan.khaus 1/1 Running 0 104s
kube-system kube-scheduler-centos01.lan.khaus 1/1 Running 0 64s
kube-system rke2-canal-9fh4g 1/2 Running 0 88s
kube-system rke2-coredns-rke2-coredns-54498b9b86-lfcbh 1/1 Running 0 88s
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-htvn2 1/1 Running 0 88s
kube-system rke2-ingress-nginx-controller-mr6xz 1/1 Running 0 62s
kube-system rke2-metrics-server-5b745c5ccb-ttfnj 1/1 Running 0 68s
[root@centos01 ~]# crictl image ls
IMAGE TAG IMAGE ID SIZE
docker.io/rancher/hardened-calico v3.20.1-build20211006 1444b9c6f8ebf 183MB
docker.io/rancher/hardened-cluster-autoscaler v1.8.5-build20211006 1b8d2fba888e1 40.8MB
docker.io/rancher/hardened-cni-plugins v0.9.1-build20210414 1a9693541e53e 62.8MB
docker.io/rancher/hardened-coredns v1.8.5-build20211006 6e03a70a0d90b 48MB
docker.io/rancher/hardened-etcd v3.5.0-k3s2-build20210915 2e6c3409c04ce 49MB
docker.io/rancher/hardened-flannel v0.14.0-build20211006 80753b7faeaa5 95.7MB
docker.io/rancher/hardened-k8s-metrics-server v0.5.0-build20210915 d94f6027f5853 47MB
docker.io/rancher/hardened-kubernetes v1.22.2-rke2r1-build20210924 ea6128c5fdee3 221MB
docker.io/rancher/hardened-multus-cni v3.7.1-build20210416 89f38b30ad002 116MB
docker.io/rancher/klipper-helm v0.6.5-build20210915 710ba7ee5997e 84.5MB
docker.io/rancher/mirrored-ingress-nginx-kube-webhook-certgen v1.0 17e55ec30f203 18.6MB
docker.io/rancher/nginx-ingress-controller nginx-1.0.2-hardened1 969b4c43d74b1 228MB
docker.io/rancher/pause 3.5 69bfc1b271447 299kB
docker.io/rancher/rke2-cloud-provider v0.0.2-build20210915 3c506666fbd5a 46.8MB
docker.io/rancher/rke2-runtime v1.22.2-dev-6f09e7cb e3fb668cf9fd4 307MB
[root@centos01 ~]# kubectl top pods -A
NAMESPACE NAME CPU(cores) MEMORY(bytes)
kube-system cloud-controller-manager-centos01.lan.khaus 6m 21Mi
kube-system etcd-centos01.lan.khaus 28m 47Mi
kube-system kube-apiserver-centos01.lan.khaus 110m 445Mi
kube-system kube-controller-manager-centos01.lan.khaus 24m 69Mi
kube-system kube-multus-ds-px2fl 2m 1Mi
kube-system kube-proxy-centos01.lan.khaus 6m 13Mi
kube-system kube-scheduler-centos01.lan.khaus 8m 26Mi
kube-system rke2-canal-9fh4g 39m 90Mi
kube-system rke2-coredns-rke2-coredns-autoscaler-567d46fb6d-htvn2 15m 10Mi
kube-system rke2-ingress-nginx-controller-mr6xz 18m 92Mi
kube-system rke2-metrics-server-5b745c5ccb-ttfnj 14m 29Mi |
canal fix: rancher/rke2-charts#169 |
Closing this out as the charts have been validated as working on v1.22.2-rc2+rke2r1 and #2022 was created for tracking vsphere and harvester. |
We need to validate that all feature charts work with Kubernetes 1.22. We also need to ensure that all the images, both core and for packaged components, are updated. This issue is to track that work.
Related issues:
Core Functionality
RKE2 Core
Update to Kubernetes / K3s 1.22.2 #1827
Bump golang versions to v1.15.15b5/v1.16.7b7 to track upstream image-build-kubernetes#16
Bump golang version to v1.16.7b7 image-build-containerd#11
Bump golang and default crictl versions image-build-crictl#4
Bump golang and default runc versions image-build-runc#7
Bump golang and k3s versions image-build-rke2-cloud-provider#7
Bump golang and default etcd version image-build-etcd#5
no update needed; the embedded helm supports 1.22
rke2-canal
Fix deprecated API in canal rke2-charts#151
Bump flannel and calico images rke2-charts#160
Bump canal chart version rke2-charts#161
bump to 1.16.7b7 build base image-build-cni-plugins#1
bump golang and package versions image-build-calico#7
Update base image and k3s-root image-build-flannel#11
Update version of base image in .drone image-build-flannel#12
rke2-coredns
Bump rke2-coredns images to v1.8.5 rke2-charts#158
Update base image image-build-coredns#8
Update the .drone file with the new base version image-build-coredns#9
Update base image image-build-coredns#8
Update the .drone file with the new base version image-build-coredns#9
Update base image image-build-dns-nodecache#3
rke2-metrics-server
Update rke2-metrics-server to v0.5.0 rke2-charts#153
Update rke2-metrics-server to v0.5.0 rke2-charts#153
rke2-ingress-nginx
Upgrade nginx controller to v1.0.2 rke2-charts#156
Add Version v1.0.2 ingress-nginx#61
Hardened nginx 1.0.2 ingress-nginx#62
Compile libmaxminddb and update lua ingress-nginx#63
fix setcap commands ingress-nginx#64
update kind version to 0.11.1 ingress-nginx#65
Fix e2e tests ingress-nginx#66
Add forked kube-webhook-certgen v1.0 image-mirror#167
https://github.com/rancherlabs/eio/issues/594
Alternative CNIs
rke2-calico
Update calico to 1.20.1 version rke2-charts#154
New image and a few version bumps image-mirror#169
rke2-cilium
cilium: Update to 1.10.4 rke2-charts#143
cilium: Fix the eni value rke2-charts#146
rke2-cilium: Disable image digest usage rke2-charts#159
cilium: Fix image URI of mount-cgroup init container rke2-charts#164
cilium: Update to 1.10.4 image-mirror#159
rke2-multus
Update multus images for 1.22 rke2-charts#163
Update the base image version image-build-multus#9
Update base image version in drone file image-build-multus#10
Update sriov-cni and base image version image-build-sriov-cni#3
rancher-sriov
[2.6.2] Update sriov to work with k8s 1.22 charts#1540
Update ib-sriov-cni and base image versions image-build-ib-sriov-cni#3
bump to 1.16.7b7 build base image-build-cni-plugins#1
Update sriov-operator and image base version image-build-sriov-operator#10
Update sriov-operator and image base version image-build-sriov-operator#10
Update sriov-operator and image base version image-build-sriov-operator#10
Update sriov-network-device-plugin and base image version image-build-sriov-network-device-plugin#5
Update sriov-network-resources-injector and base image version image-build-sriov-network-resources-injector#5
Cloud Providers
rancher-vsphere-cpi and rancher-vsphere-csi
rancher-vsphere-cpi and rancher-vsphere-csi
The text was updated successfully, but these errors were encountered: