Problems that occur when deploying kind using Docker under CentOS 7.9 version

[gitlayzer]

First some versions of mine are

os：CentOS 7.9
kind：0.20.0
docker：24.0.7
containerd：1.6.25
kernel：5.4.256-1.el7.elrepo.x86_64

kind config

apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
networking:
disableDefaultCNI: true
podSubnet: "10.244.0.0/16"
serviceSubnet: "10.96.0.0/16"
nodes:

• role: control-plane
• role: worker
• role: worker

Problems

Creating cluster "calico-ipip" ...
✓ Ensuring node image (docker.devops-engineer.com.cn/kindest/node:v1.26.6) 🖼
✓ Preparing nodes 📦 📦 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹
✓ Installing StorageClass 💾
✗ Joining worker nodes 🚜
Deleted nodes: ["calico-ipip-worker2" "calico-ipip-control-plane" "calico-ipip-worker"]
ERROR: failed to create cluster: failed to join node with kubeadm: command "docker exec --privileged calico-ipip-worker kubeadm join --config /kind/kubeadm.conf --skip-phases=preflight --v=6" failed with error: exit status 1
Command Output: I1201 05:34:07.973943 199 join.go:408] [preflight] found NodeName empty; using OS hostname as NodeName
I1201 05:34:07.974011 199 joinconfiguration.go:76] loading configuration from "/kind/kubeadm.conf"
I1201 05:34:07.974599 199 controlplaneprepare.go:225] [download-certs] Skipping certs download
I1201 05:34:07.974620 199 join.go:525] [preflight] Discovering cluster-info
I1201 05:34:07.974628 199 token.go:80] [discovery] Created cluster-info discovery client, requesting info from "calico-ipip-control-plane:6443"
I1201 05:34:07.980457 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 5 milliseconds
I1201 05:34:07.980678 199 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
I1201 05:34:13.890005 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 1 milliseconds
I1201 05:34:13.890212 199 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
I1201 05:34:20.306721 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 1 milliseconds
I1201 05:34:20.306816 199 token.go:223] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
I1201 05:34:26.306582 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 1 milliseconds
I1201 05:34:26.307269 199 token.go:105] [discovery] Cluster info signature and contents are valid and no TLS pinning was specified, will use API Server "calico-ipip-control-plane:6443"
I1201 05:34:26.307293 199 discovery.go:52] [discovery] Using provided TLSBootstrapToken as authentication credentials for the join process
I1201 05:34:26.307300 199 join.go:539] [preflight] Fetching init configuration
I1201 05:34:26.307302 199 join.go:585] [preflight] Retrieving KubeConfig objects
I1201 05:34:26.312413 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 4 milliseconds
I1201 05:34:26.315235 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kube-proxy?timeout=10s 200 OK in 2 milliseconds
I1201 05:34:26.316037 199 kubelet.go:74] attempting to download the KubeletConfiguration from ConfigMap "kubelet-config"
I1201 05:34:26.317114 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/namespaces/kube-system/configmaps/kubelet-config?timeout=10s 200 OK in 0 milliseconds
I1201 05:34:26.318246 199 interface.go:432] Looking for default routes with IPv4 addresses
I1201 05:34:26.318251 199 interface.go:437] Default route transits interface "eth0"
I1201 05:34:26.318345 199 interface.go:209] Interface eth0 is up
I1201 05:34:26.318373 199 interface.go:257] Interface "eth0" has 3 addresses :[172.18.0.4/16 fc00:f853:ccd:e793::4/64 fe80::42:acff:fe12:4/64].
I1201 05:34:26.318389 199 interface.go:224] Checking addr 172.18.0.4/16.
I1201 05:34:26.318393 199 interface.go:231] IP found 172.18.0.4
I1201 05:34:26.318397 199 interface.go:263] Found valid IPv4 address 172.18.0.4 for interface "eth0".
I1201 05:34:26.318400 199 interface.go:443] Found active IP 172.18.0.4
I1201 05:34:26.322006 199 kubelet.go:121] [kubelet-start] writing bootstrap kubelet config file at /etc/kubernetes/bootstrap-kubelet.conf
I1201 05:34:26.322393 199 kubelet.go:136] [kubelet-start] writing CA certificate at /etc/kubernetes/pki/ca.crt
I1201 05:34:26.322597 199 loader.go:373] Config loaded from file: /etc/kubernetes/bootstrap-kubelet.conf
I1201 05:34:26.322756 199 kubelet.go:157] [kubelet-start] Checking for an existing Node in the cluster with name "calico-ipip-worker" and status "Ready"
I1201 05:34:26.324309 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/nodes/calico-ipip-worker?timeout=10s 404 Not Found in 1 milliseconds
I1201 05:34:26.324622 199 kubelet.go:172] [kubelet-start] Stopping the kubelet
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
I1201 05:34:27.568594 199 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I1201 05:34:27.568960 199 cert_rotation.go:137] Starting client certificate rotation controller
I1201 05:34:27.569105 199 loader.go:373] Config loaded from file: /etc/kubernetes/kubelet.conf
I1201 05:34:27.569290 199 kubelet.go:220] [kubelet-start] preserving the crisocket information for the node
I1201 05:34:27.569321 199 patchnode.go:31] [patchnode] Uploading the CRI Socket information "unix:///run/containerd/containerd.sock" to the Node API object "calico-ipip-worker" as an annotation
I1201 05:34:28.076743 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/nodes/calico-ipip-worker?timeout=10s 404 Not Found in 6 milliseconds
I1201 05:34:28.570605 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/nodes/calico-ipip-worker?timeout=10s 404 Not Found in 1 milliseconds
I1201 05:34:29.071753 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/nodes/calico-ipip-worker?timeout=10s 404 Not Found in 1 milliseconds
I1201 05:34:29.573142 199 round_trippers.go:553] GET https://calico-ipip-control-plane:6443/api/v1/nodes/calico-ipip-worker?timeout=10s 404 Not Found in 1 milliseconds
......
nodes "calico-ipip-worker" not found
error uploading crisocket
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/join.runKubeletStartJoinPhase
cmd/kubeadm/app/cmd/phases/join/kubelet.go:222
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
cmd/kubeadm/app/cmd/phases/workflow/runner.go:259
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdJoin.func1
cmd/kubeadm/app/cmd/join.go:180
github.com/spf13/cobra.(*Command).execute
vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
cmd/kubeadm/app/kubeadm.go:50
main.main
cmd/kubeadm/kubeadm.go:25
runtime.main
/usr/local/go/src/runtime/proc.go:250
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594
error execution phase kubelet-start
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run.func1
cmd/kubeadm/app/cmd/phases/workflow/runner.go:260
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).visitAll
cmd/kubeadm/app/cmd/phases/workflow/runner.go:446
k8s.io/kubernetes/cmd/kubeadm/app/cmd/phases/workflow.(*Runner).Run
cmd/kubeadm/app/cmd/phases/workflow/runner.go:232
k8s.io/kubernetes/cmd/kubeadm/app/cmd.newCmdJoin.func1
cmd/kubeadm/app/cmd/join.go:180
github.com/spf13/cobra.(*Command).execute
vendor/github.com/spf13/cobra/command.go:916
github.com/spf13/cobra.(*Command).ExecuteC
vendor/github.com/spf13/cobra/command.go:1040
github.com/spf13/cobra.(*Command).Execute
vendor/github.com/spf13/cobra/command.go:968
k8s.io/kubernetes/cmd/kubeadm/app.Run
cmd/kubeadm/app/kubeadm.go:50
main.main
cmd/kubeadm/kubeadm.go:25
runtime.main
/usr/local/go/src/runtime/proc.go:250
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1594

[gitlayzer]

@BenTheElder

[gitlayzer]

However, I solved this problem using Ubuntu 22.04.4 LTS

[BenTheElder]

@BenTheElder

nit: please just file issue and we'll triage them as we get to them between us 😅

To debug this we need the log export from the instructions in the issue template. The logs from the command itself tell us api server didn't come up healthy but to determine why we need to inspect the cluster state.

[BenTheElder]

this seems like a duplicate of #3311

[gitlayzer]

这似乎是#3311的重复

OK thank you, I understand.