Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-1.27] DaemonSet for ContainerNetworking DHCP CNI Plugin #5208

Closed
thomasferrandiz opened this issue Jan 5, 2024 · 1 comment
Closed

[release-1.27] DaemonSet for ContainerNetworking DHCP CNI Plugin #5208

thomasferrandiz opened this issue Jan 5, 2024 · 1 comment
Assignees
@endawkins
Milestone
v1.27.10+rke2r1

Comments

@thomasferrandiz
Copy link
Contributor

Backport of: #3917
@thomasferrandiz thomasferrandiz added this to the v1.27.10+rke2r1 milestone Jan 5, 2024
@thomasferrandiz thomasferrandiz mentioned this issue Jan 5, 2024
Merged
@endawkins
Copy link

Validated on 1.27 with ab61d33 / version 1.27

Environment Details

Infrastructure

  • Cloud
  • Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-24-94 6.2.0-1017-aws #17~22.04.1-Ubuntu SMP Fri Nov 17 21:07:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Cluster Configuration:

1 server
1 agent

Config.yaml:

write-kubeconfig-mode: 644
token: test
node-external-ip: <EXTERNAL_IP>
cni: multus,calico

Additional files

***rke2-multus-config.yaml

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-multus
  namespace: kube-system
spec:
  valuesContent: |-
    manifests:
      dhcpDaemonSet: true

Testing Steps

  1. Copy config.yaml
$ sudo mkdir -p /etc/rancher/rke2 && sudo cp config.yaml /etc/rancher/rke2
  1. Install RKE2 (Do not enable or start rke2)
  2. Create file /var/lib/rancher/rke2/server/manifests/rke2-multus-config.yaml
  3. Enable and start rke2 (both server and agent)
  4. Verify that kube-rke2-multus-dhcp was created and started properly
  5. Verify /run/cni/dhcp.sock was created on agent node

Replication Results:

  • rke2 version used for replication:
$ rke2 -v
rke2 version v1.27.9+rke2r1 (378bd59c4f0f97094c23c350d668f37f33aba406)
go version go1.20.12 X:boringcrypto

$ kubectl get nodes,pods -A -o wide                                                                                                                                                       ip-172-31-30-106: Wed Jan 24 18:02:13 2024

NAME                    STATUS   ROLES                       AGE     VERSION          INTERNAL-IP     EXTERNAL-IP     OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
node/ip-172-31-24-197   Ready    <none>                      2m20s   v1.27.9+rke2r1   172.31.24.197   <none>          Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2
node/ip-172-31-30-106   Ready    control-plane,etcd,master   4m40s   v1.27.9+rke2r1   172.31.30.106   <EXTERNAL_IP>   Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2

NAMESPACE         NAME                                                       READY   STATUS      RESTARTS        AGE     IP              NODE               NOMINATED NODE   READINESS GATES
calico-system     pod/calico-kube-controllers-7bdcf6759c-88q8t               1/1     Running     0               3m42s   10.42.17.131    ip-172-31-30-106   <none>           <none>
calico-system     pod/calico-node-tjqwm                                      1/1     Running     0               2m20s   172.31.24.197   ip-172-31-24-197   <none>           <none>
calico-system     pod/calico-node-zzk26                                      1/1     Running     0               3m42s   172.31.30.106   ip-172-31-30-106   <none>           <none>
calico-system     pod/calico-typha-66ff56c5c6-fjmgs                          1/1     Running     0               3m42s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/cloud-controller-manager-ip-172-31-30-106              1/1     Running     0               4m25s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/etcd-ip-172-31-30-106                                  1/1     Running     0               4m20s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-calico-25jcn                         0/1     Completed   2               4m20s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-calico-crd-p2t2r                     0/1     Completed   0               4m20s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-coredns-4tqlm                        0/1     Completed   0               4m20s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-ingress-nginx-g7rhn                  0/1     Completed   0               4m20s   10.42.17.132    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-metrics-server-j4l5d                 0/1     Completed   0               4m20s   10.42.17.133    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-multus-mvftv                         0/1     Completed   0               4m20s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-87k7p            0/1     Completed   1               4m20s   10.42.17.136    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-crd-t945w        0/1     Completed   0               4m20s   10.42.17.130    ip-172-31-30-106   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-validation-webhook-qx6jr    0/1     Completed   0               4m20s   10.42.17.129    ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-apiserver-ip-172-31-30-106                        1/1     Running     0               4m34s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-controller-manager-ip-172-31-30-106               1/1     Running     0               4m28s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-24-197                            1/1     Running     0               2m20s   172.31.24.197   ip-172-31-24-197   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-30-106                            1/1     Running     0               4m31s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/kube-scheduler-ip-172-31-30-106                        1/1     Running     0               4m26s   172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-autoscaler-6f97df447-8vbr2   1/1     Running     0               4m6s    10.42.17.134    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-f6c9f9649-2xklf              1/1     Running     0               2m18s   10.42.243.128   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-f6c9f9649-pp4dh              1/1     Running     0               4m6s    10.42.17.135    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-6d5xp                    1/1     Running     0               113s    10.42.243.129   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-p6wkp                    1/1     Running     0               2m36s   10.42.17.141    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-metrics-server-6d79d977db-qvp6m                   1/1     Running     0               2m50s   10.42.17.138    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-multus-ds-74z9s                                   1/1     Running     2 (3m41s ago)   4m7s    172.31.30.106   ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-multus-ds-g8zlz                                   1/1     Running     0               2m18s   172.31.24.197   ip-172-31-24-197   <none>           <none>
kube-system       pod/rke2-snapshot-controller-7d6476d7cb-2drzm              1/1     Running     0               2m41s   10.42.17.140    ip-172-31-30-106   <none>           <none>
kube-system       pod/rke2-snapshot-validation-webhook-5649fbd66c-tl28v      1/1     Running     0               2m52s   10.42.17.137    ip-172-31-30-106   <none>           <none>
tigera-operator   pod/tigera-operator-5b8fcdd5f6-xfxkz                       1/1     Running     0               3m49s   172.31.30.106   ip-172-31-30-106   <none>           <none>

$ ls -l /run/cni/
ls: cannot access '/run/cni/': No such file or directory

Validation Results:

  • rke2 version used for validation:
$ rke2 -v
rke2 version v1.27.10-rc2+rke2r1 (ab61d3304744a88ecf23e02df124f845d246c3fe)
go version go1.20.13 X:boringcrypto

$ kubectl get nodes,pods -A -o wide                                                                                                                                                        ip-172-31-24-94: Wed Jan 24 19:00:13 2024

NAME                   STATUS   ROLES                       AGE     VERSION           INTERNAL-IP    EXTERNAL-IP     OS-IMAGE             KERNEL-VERSION   CONTAINER-RUNTIME
node/ip-172-31-24-94   Ready    control-plane,etcd,master   4m40s   v1.27.10+rke2r1   172.31.24.94   <EXTERNAL_IP>   Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2
node/ip-172-31-31-6    Ready    <none>                      93s     v1.27.10+rke2r1   172.31.31.6    <none>          Ubuntu 22.04.3 LTS   6.2.0-1017-aws   containerd://1.7.11-k3s2

NAMESPACE         NAME                                                        READY   STATUS      RESTARTS        AGE     IP              NODE              NOMINATED NODE   READINESS GATES
calico-system     pod/calico-kube-controllers-67db698866-t4q4c                1/1     Running     0               3m32s   10.42.186.133   ip-172-31-24-94   <none>           <none>
calico-system     pod/calico-node-l29m2                                       1/1     Running     0               93s     172.31.31.6     ip-172-31-31-6    <none>           <none>
calico-system     pod/calico-node-w2jmz                                       1/1     Running     0               3m32s   172.31.24.94    ip-172-31-24-94   <none>           <none>
calico-system     pod/calico-typha-76d6444ddc-jfjk9                           1/1     Running     0               3m32s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/cloud-controller-manager-ip-172-31-24-94                1/1     Running     0               4m15s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/etcd-ip-172-31-24-94                                    1/1     Running     0               3m40s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-calico-crd-mlwqk                      0/1     Completed   0               4m22s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-calico-wpnk6                          0/1     Completed   0               4m22s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-coredns-c9f7t                         0/1     Completed   0               4m22s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-ingress-nginx-q8fcv                   0/1     Completed   0               4m22s   10.42.186.136   ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-metrics-server-7gzxk                  0/1     Completed   0               4m22s   10.42.186.132   ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-multus-gpn9v                          0/1     Completed   0               4m22s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-crd-mf4v6         0/1     Completed   0               4m22s   10.42.186.134   ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-controller-mtrs4             0/1     Completed   1               4m22s   10.42.186.130   ip-172-31-24-94   <none>           <none>
kube-system       pod/helm-install-rke2-snapshot-validation-webhook-rszcq     0/1     Completed   0               4m22s   10.42.186.129   ip-172-31-24-94   <none>           <none>
kube-system       pod/kube-apiserver-ip-172-31-24-94                          1/1     Running     0               3m58s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/kube-controller-manager-ip-172-31-24-94                 1/1     Running     0               3m51s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-24-94                              1/1     Running     0               4m10s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/kube-proxy-ip-172-31-31-6                               1/1     Running     0               93s     172.31.31.6     ip-172-31-31-6    <none>           <none>
kube-system       pod/kube-scheduler-ip-172-31-24-94                          1/1     Running     0               3m56s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-84f49dccc9-6r7jd              0/1     Running     0               86s     10.42.126.1     ip-172-31-31-6    <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-84f49dccc9-f65vw              1/1     Running     0               3m53s   10.42.186.135   ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-coredns-rke2-coredns-autoscaler-5b5b56997b-qvqgf   1/1     Running     0               3m53s   10.42.186.131   ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-7xspx                     0/1     Running     0               43s     10.42.126.2     ip-172-31-31-6    <none>           <none>
kube-system       pod/rke2-ingress-nginx-controller-dmjfm                     1/1     Running     0               2m20s   10.42.186.141   ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-metrics-server-5c9768ff67-rnlbs                    1/1     Running     0               2m37s   10.42.186.138   ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-multus-bjkn8                                       1/1     Running     3 (3m16s ago)   3m55s   172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-multus-dhcp-7rktv                                  1/1     Running     0               3m8s    172.31.24.94    ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-multus-dhcp-w4pxc                                  1/1     Running     0               43s     172.31.31.6     ip-172-31-31-6    <none>           <none>
kube-system       pod/rke2-multus-hhzj9                                       1/1     Running     3 (52s ago)     93s     172.31.31.6     ip-172-31-31-6    <none>           <none>
kube-system       pod/rke2-snapshot-controller-7d6476d7cb-rhdw8               1/1     Running     0               2m30s   10.42.186.139   ip-172-31-24-94   <none>           <none>
kube-system       pod/rke2-snapshot-validation-webhook-5649fbd66c-xqb76       1/1     Running     0               2m41s   10.42.186.137   ip-172-31-24-94   <none>           <none>
tigera-operator   pod/tigera-operator-5b8fcdd5f6-4nwnw                        1/1     Running     0               3m39s   172.31.24.94    ip-172-31-24-94   <none>           <none>

$ ls -l /run/cni/
total 0
srwxr-xr-x 1 root root 0 Jan 24 18:59 dhcp.sock

Additional context / logs:

N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@endawkins endawkins

Labels
None yet
Projects
None yet
Milestone
v1.27.10+rke2r1
Development

No branches or pull requests
2 participants
@thomasferrandiz @endawkins