level=fatal msg="host IPv6 unknown" #5922

lks-hrsch · 2022-07-31T16:15:26Z

lks-hrsch
Jul 31, 2022

Hi k3s community, I currently have issues joining k3s-agents to my cluster.
I get the following log error: level=fatal msg="host IPv6 unknown".
One month ago, this Setup works totally fine, but now it doesn't work anymore.

About my Cluster

3 Master with ETCD on premise these are x64 nodes and
2 Workers at my home (Raspberry Pis).
These are all connected through a WireGuard network, and I am trying to create a dual-stack cluster.

NAME      STATUS     ROLES                       AGE   VERSION
cancer    Ready      control-plane,etcd,master   51m   v1.24.3+k3s1
earth     NotReady   <none>                      48m   v1.24.3+k3s1
gemini    Ready      control-plane,etcd,master   50m   v1.24.3+k3s1
mercury   NotReady   <none>                      50m   v1.24.3+k3s1
taurus    Ready      control-plane,etcd,master   57m   v1.24.3+k3s1

About the worker mercury (Raspberry Pi 4)

pi@mercury:~ $ uname -a
Linux mercury 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64 GNU/Linux

journalctl -xe -u k3s-agent

pi@mercury:~ $ journalctl -xe -u k3s-agent

Jul 31 17:49:57 mercury systemd[1]: Starting Lightweight Kubernetes...
░░ Subject: A start job for unit k3s-agent.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s-agent.service has begun execution.
░░ 
░░ The job identifier is 32025.
Jul 31 17:49:57 mercury sh[42078]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Jul 31 17:49:57 mercury sh[42079]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Jul 31 17:49:57 mercury k3s[42082]: time="2022-07-31T17:49:57+02:00" level=info msg="Starting k3s agent v1.24.3+k3s1 (990ba0e8)"
Jul 31 17:49:57 mercury k3s[42082]: time="2022-07-31T17:49:57+02:00" level=info msg="Running load balancer k3s-agent-load-balancer xxx.xxx.xxx.xxx:6444 -> [xxx.xxx.xxx.xxx:6443 xxx.xxx.xxx.xxx:6443 xxx.xxx.xxx.xxx:6443 xxx.xxx.xxx.xxx:6443]"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=info msg="Module overlay was already loaded"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=info msg="Module nf_conntrack was already loaded"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=info msg="Module br_netfilter was already loaded"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=info msg="Module iptable_nat was already loaded"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=info msg="Module ip6table_nat was already loaded"
Jul 31 17:49:58 mercury k3s[42082]: W0731 17:49:58.670320   42082 sysinfo.go:203] Nodes topology is not available, providing CPU topology
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="getConntrackMax: using conntrack-min"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="Creating the CNI conf in directory /var/lib/rancher/k3s/agent/etc/cni/net.d"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="Creating the flannel configuration for backend vxlan in file /var/lib/rancher/k3s/agent/etc/flannel/net-conf.json"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="The flannel configuration is {\n\t\"Network\": \"xxx.xxx.xxx.xxx/16\",\n\t\"EnableIPv6\": true,\n\t\"EnableIPv4\": true,\n\t\"IPv6Network\": \"fd42::/48\",\n\t\"Backend\": {\n\t\"Type\": \"vxlan\"\n}\>
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="Searching for nvidia container runtime at /usr/local/nvidia/toolkit/nvidia-container-runtime"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="nvidia container runtime not found at /usr/local/nvidia/toolkit/nvidia-container-runtime"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="Searching for nvidia container runtime at /usr/bin/nvidia-container-runtime"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="nvidia container runtime not found at /usr/bin/nvidia-container-runtime"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="Searching for nvidia-experimental container runtime at /usr/local/nvidia/toolkit/nvidia-container-runtime-experimental"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="nvidia-experimental container runtime not found at /usr/local/nvidia/toolkit/nvidia-container-runtime-experimental"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="Searching for nvidia-experimental container runtime at /usr/bin/nvidia-container-runtime-experimental"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=debug msg="nvidia-experimental container runtime not found at /usr/bin/nvidia-container-runtime-experimental"
Jul 31 17:49:58 mercury k3s[42082]: time="2022-07-31T17:49:58+02:00" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58+02:00" level=warning msg="containerd config version `1` has been deprecated and will be removed in containerd v2.0, please switch to version `2`, see https://github.com/containerd/containerd/blob/main/docs/PLUGINS.md#v>
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.749913799+02:00" level=info msg="starting containerd" revision= version=v1.6.6-k3s1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.803267947+02:00" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.803462927+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.809418701+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /li>
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.809545607+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.809936253+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to >
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.810011919+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.810061289+02:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.810100307+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.810187566+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.810715414+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.fuse-overlayfs\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.810826747+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.stargz\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.811824795+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812168441+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with >
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812242718+02:00" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812307310+02:00" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812350014+02:00" level=info msg="metadata content store policy set" policy=shared
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812586568+02:00" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812664641+02:00" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812712326+02:00" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812814362+02:00" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812865344+02:00" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812916565+02:00" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.812962917+02:00" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.813012787+02:00" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.813059416+02:00" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.813108860+02:00" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.813314507+02:00" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.813469969+02:00" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.813772171+02:00" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.814053021+02:00" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.814949866+02:00" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815075736+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815130995+02:00" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815310271+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815377771+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815443178+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815492029+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815544418+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815593047+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815663065+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815709528+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815763472+02:00" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.815970841+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816035692+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816085006+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816127969+02:00" level=info msg="loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." type=io.containerd.tracing.processor.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816180043+02:00" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="no OpenTelemetry endpoint: skip plugin" type=io.containerd.tracing.processor.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816226580+02:00" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816285153+02:00" level=error msg="failed to initialize a tracing processor \"otlp\"" error="no OpenTelemetry endpoint: skip plugin"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816401856+02:00" level=info msg="loading plugin \"io.containerd.grpc.v1.cri\"..." type=io.containerd.grpc.v1
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.816977537+02:00" level=info msg="Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntimeName:runc DefaultRuntime:{Type: Path: Engine: PodAnnotations:[] ContainerAnnotations:[]>
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.817202351+02:00" level=info msg="Connect containerd service"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.817329998+02:00" level=info msg="Get image filesystem path \"/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlayfs\""
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.818994134+02:00" level=info msg="Start subscribing containerd event"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819178355+02:00" level=info msg="Start recovering state"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819440131+02:00" level=info msg="Start event monitor"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819540168+02:00" level=info msg="Start snapshots syncer"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819547908+02:00" level=info msg=serving... address=/run/k3s/containerd/containerd.sock.ttrpc
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819583778+02:00" level=info msg="Start cni network conf syncer for default"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819616704+02:00" level=info msg="Start streaming server"
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819754851+02:00" level=info msg=serving... address=/run/k3s/containerd/containerd.sock
Jul 31 17:49:58 mercury k3s[42096]: time="2022-07-31T17:49:58.819823888+02:00" level=info msg="containerd successfully booted in 0.071590s"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Containerd is now running"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Getting list of apiserver endpoints from server"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Updating load balancer k3s-agent-load-balancer default server address -> xxx.xxx.xxx.xxx:6443"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Connecting to proxy" url="wss://xxx.xxx.xxx.xxx:6443/v1-k3s/connect"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Connecting to proxy" url="wss://xxx.xxx.xxx.xxx:6443/v1-k3s/connect"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=debug msg="Kubelet image credential provider bin directory check failed: stat /var/lib/rancher/credentialprovider/bin: no such file or directory"
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Running kubelet --address=xxx.xxx.xxx.xxx --allowed-unsafe-sysctls=net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=sys>
Jul 31 17:49:59 mercury k3s[42082]: time="2022-07-31T17:49:59+02:00" level=info msg="Connecting to proxy" url="wss://xxx.xxx.xxx.xxx:6443/v1-k3s/connect"
Jul 31 17:50:00 mercury k3s[42082]: Flag --cloud-provider has been deprecated, will be removed in 1.24 or later, in favor of removing cloud provider code from Kubelet.
Jul 31 17:50:00 mercury k3s[42082]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Jul 31 17:50:00 mercury k3s[42082]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.27. Image garbage collector will get sandbox image information from CRI.
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.047826   42082 server.go:192] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=info msg="Annotations and labels have already set on node: mercury"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.102917   42082 server.go:395] "Kubelet version" kubeletVersion="v1.24.3+k3s1"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.103448   42082 server.go:397] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.107120   42082 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt"
Jul 31 17:50:00 mercury k3s[42082]: W0731 17:50:00.107645   42082 manager.go:159] Cannot detect current cgroup on cgroup v2
Jul 31 17:50:00 mercury k3s[42082]: W0731 17:50:00.111344   42082 sysinfo.go:203] Nodes topology is not available, providing CPU topology
Jul 31 17:50:00 mercury k3s[42082]: W0731 17:50:00.115137   42082 machine.go:65] Cannot read vendor id correctly, set empty.
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.118876   42082 server.go:644] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.119825   42082 container_manager_linux.go:262] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.120035   42082 container_manager_linux.go:267] "Creating Container Manager object based on Node Config" nodeConfig={RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: KubeletOOMScoreAdj:-999 ContainerRuntime: CgroupsPerQO>
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.120341   42082 topology_manager.go:133] "Creating topology manager with policy per scope" topologyPolicyName="none" topologyScopeName="container"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.120586   42082 container_manager_linux.go:302] "Creating device plugin manager" devicePluginEnabled=true
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.120784   42082 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=info msg="Starting flannel with backend vxlan"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.133367   42082 kubelet.go:376] "Attempting to sync node with API server"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.133464   42082 kubelet.go:267] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.134180   42082 kubelet.go:278] "Adding apiserver pod source"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.134300   42082 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.136168   42082 kuberuntime_manager.go:239] "Container runtime initialized" containerRuntime="containerd" version="v1.6.6-k3s1" apiVersion="v1"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.137164   42082 server.go:1177] "Started kubelet"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.137608   42082 server.go:150] "Starting to listen" address="xxx.xxx.xxx.xxx" port=10250
Jul 31 17:50:00 mercury k3s[42082]: E0731 17:50:00.139683   42082 cri_stats_provider.go:455] "Failed to get the info of the filesystem with mountpoint" err="unable to find data in memory cache" mountpoint="/var/lib/rancher/k3s/agent/containerd/io.containerd.snapshotter.v1.overlay>
Jul 31 17:50:00 mercury k3s[42082]: E0731 17:50:00.143015   42082 kubelet.go:1298] "Image garbage collection failed once. Stats initialization may not have completed yet" err="invalid capacity 0 on image filesystem"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.142152   42082 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.142304   42082 server.go:410] "Adding debug handlers to kubelet server"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.155092   42082 volume_manager.go:289] "Starting Kubelet Volume Manager"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.157993   42082 desired_state_of_world_populator.go:145] "Desired state populator starts to run"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dbytes=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dios=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=info msg="Flannel found PodCIDR assigned for node mercury"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="The interface wg0 will be used by flannel"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=info msg="The interface wg0 with ipv4 address xxx.xxx.xxx.xxx will be used by flannel"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=info msg="Using dual-stack mode. The ipv6 address f6bb:293e:23ac::aaaa:5 will be used by flannel"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dbytes=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dios=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.189836   42082 kube.go:121] Waiting 10m0s for node controller to sync
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.189905   42082 kube.go:402] Starting kube subnet manager
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.223937   42082 kube.go:430] Creating the node lease for IPv6. This is the n.Spec.PodCIDRs: [xxx.xxx.xxx.xxx/24 fd42:0:0:2::/64]
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.224060   42082 kube.go:430] Creating the node lease for IPv6. This is the n.Spec.PodCIDRs: [xxx.xxx.xxx.xxx/24 fd42:0:0:1::/64]
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.224110   42082 kube.go:430] Creating the node lease for IPv6. This is the n.Spec.PodCIDRs: [xxx.xxx.xxx.xxx/24 fd42::/64]
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dbytes=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dios=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.255930   42082 cpu_manager.go:213] "Starting CPU manager" policy="none"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.255982   42082 cpu_manager.go:214] "Reconciling" reconcilePeriod="10s"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.256041   42082 state_mem.go:36] "Initialized new in-memory state store"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.256537   42082 state_mem.go:88] "Updated default CPUSet" cpuSet=""
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.256618   42082 state_mem.go:96] "Updated CPUSet assignments" assignments=map[]
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.256668   42082 policy_none.go:49] "None policy: Start"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.257883   42082 kubelet_network_linux.go:76] "Initialized protocol iptables rules." protocol=IPv4
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.258303   42082 kubelet_node_status.go:70] "Attempting to register node" node="mercury"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.259397   42082 memory_manager.go:168] "Starting memorymanager" policy="None"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.259479   42082 state_mem.go:35] "Initializing new in-memory state store"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.260012   42082 state_mem.go:75] "Updated machine memory state"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.262079   42082 kuberuntime_manager.go:1095] "Updating runtime config through cri with podcidr" CIDR="xxx.xxx.xxx.xxx/24,fd42:0:0:3::/64"
Jul 31 17:50:00 mercury k3s[42096]: time="2022-07-31T17:50:00.264253359+02:00" level=info msg="No cni config template is specified, wait for other system components to drop the config."
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.266624   42082 kubelet_network.go:60] "Updating Pod CIDR" originalPodCIDR="" newPodCIDR="xxx.xxx.xxx.xxx/24,fd42:0:0:3::/64"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=info msg="Running kube-proxy --cluster-cidr=xxx.xxx.xxx.xxx/16,fd42::/48 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wait=0s --conntrack-tcp-timeout-established=0s --healthz-bind-address=xxx.xxx.xxx.xxx ->
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.286484   42082 server.go:231] "Warning, all flags other than --config, --write-config-to, and --cleanup are deprecated, please begin using a config file ASAP"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.286501   42082 manager.go:610] "Failed to read data from checkpoint" checkpoint="kubelet_internal_checkpoint" err="checkpoint is not found"
Jul 31 17:50:00 mercury k3s[42082]: I0731 17:50:00.290147   42082 plugin_manager.go:114] "Starting Kubelet Plugin Manager"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dbytes=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=debug msg="cgroupv2 io stats: skipping over unmappable dios=0 entry"
Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=fatal msg="host IPv6 unknown"
Jul 31 17:50:00 mercury systemd[1]: k3s-agent.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit k3s-agent.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Jul 31 17:50:00 mercury systemd[1]: k3s-agent.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s-agent.service has entered the 'failed' state with result 'exit-code'.
Jul 31 17:50:00 mercury systemd[1]: k3s-agent.service: Unit process 42139 (ip6tables) remains running after unit stopped.
Jul 31 17:50:00 mercury systemd[1]: Failed to start Lightweight Kubernetes.

About the worker earth (Raspberry Pi 3)

pi@earth:~ $ uname -a
Linux earth 5.15.55-v8+ #1573 SMP PREEMPT Mon Jul 18 17:47:52 BST 2022 aarch64 GNU/Linux

The journalctl log is the same.

How I setup the cluster

Setup server

- name: get and install k3s as server (join)
  shell: >
    curl -sfL https://get.k3s.io |
    K3S_URL=https://xxx.xxx.xxx.xxx:6443 K3S_TOKEN="{{ k3s_token }}"
    sh -s - server
    --tls-san xxx.xxx.xxx
    --disable traefik --disable servicelb --disable metrics-server
    --kubelet-arg=resolv-conf=/run/systemd/resolve/resolv.conf
    --cluster-cidr=10.42.0.0/16,fd42::/48 --service-cidr=10.43.0.0/16,fd43::/112
    --node-ip={{ node_ip }},{{ node_ipv6 }}  --node-external-ip={{ node_external_ip }},{{ node_external_ipv6 }}
    --flannel-iface=wg0
    --debug
  register: k3s_output
  ignore_errors: true
- debug: var=k3s_output

Setup agent

- name: get and install k3s as agent (join)
  shell: > 
    curl -sfL https://get.k3s.io |
    K3S_URL=https://xxx.xxx.xxx.xxx:6443 K3S_TOKEN="{{ k3s_token }}"
    sh -s - agent
    --node-ip={{ node_ip }},{{ node_ipv6 }}
    --flannel-iface=wg0
    --debug
  register: k3s_output
  ignore_errors: true
- debug: var=k3s_output

Do you see any obvious mistakes i made?

Steps I tried out

use older versions of k3s
try to set it up with docker
use a mysql as datastore-endpoint

I censored some IPs with a regex search.

brandond · 2022-08-01T21:34:01Z

brandond
Aug 1, 2022
Collaborator

Jul 31 17:50:00 mercury k3s[42082]: time="2022-07-31T17:50:00+02:00" level=fatal msg="host IPv6 unknown"

This error appears to come from kube-router network policy engine. It appears that this node does not have an IPv6 address assigned, despite the cluster and node both being configured for dual-stack:
https://github.com/k3s-io/kube-router/blob/v1.5.0+k3s2/pkg/utils/node.go#L118

Can you show the output of kubectl get node mercury -o yaml?

As a work-around, you can disable kube-router's network policy engine by starting the servers with the --disable-network-policy flag.

4 replies

lks-hrsch Aug 4, 2022
Author

Hi @brandond,

this was definitely the answer for my problem.

before the `--disable-network-policy` flag

kubectl get node mercury -o yaml

❯ kubectl get node mercury -o yaml
apiVersion: v1
kind: Node
metadata:
  annotations:
    alpha.kubernetes.io/provided-node-ip: xxx.xxx.xxx.xxx
    k3s.io/hostname: mercury
    k3s.io/internal-ip: xxx.xxx.xxx.xxx,xxxx:xxxx:xxxx::xxxx:xxxx
    k3s.io/node-args: '["agent","--node-ip","xxx.xxx.xxx.xxx,xxxx:xxxx:xxxx::xxxx:xxxx","--flannel-iface","wg0","--debug"]'
    k3s.io/node-config-hash: WAGOFD7X3Q77AWWJ74YKEYBKC2DYX72ACENGVZLJDSCBUJIRTR5A====
    k3s.io/node-env: '{"K3S_DATA_DIR":"/var/lib/rancher/k3s/data/230cb7b95d48672d3d564dfde9c7b6230bccc82b2a3298f565dc3cca2a06b2eb","K3S_TOKEN":"********","K3S_URL":"https://xxx.xxx.xxx.xxx:6443"}'
    node.alpha.kubernetes.io/ttl: "0"
    volumes.kubernetes.io/controller-managed-attach-detach: "true"
  creationTimestamp: "2022-08-02T16:33:39Z"
  finalizers:
  - wrangler.cattle.io/node
  - wrangler.cattle.io/managed-etcd-controller
  labels:
    beta.kubernetes.io/arch: arm64
    beta.kubernetes.io/instance-type: k3s
    beta.kubernetes.io/os: linux
    egress.k3s.io/cluster: "true"
    kubernetes.io/arch: arm64
    kubernetes.io/hostname: mercury
    kubernetes.io/os: linux
    node.kubernetes.io/instance-type: k3s
  name: mercury
  resourceVersion: "580757"
  uid: d6e21323-df50-472d-8d90-be28dabaae5a
spec:
  podCIDR: 10.42.3.0/24
  podCIDRs:
  - 10.42.3.0/24
  - fd42:0:0:3::/64
  providerID: k3s://mercury
status:
  addresses:
  - address: xxx.xxx.xxx.xxx
    type: InternalIP
  - address: mercury
    type: Hostname
  allocatable:
    cpu: "4"
    ephemeral-storage: "467417115077"
    memory: 3884532Ki
    pods: "110"
  capacity:
    cpu: "4"
    ephemeral-storage: 480486344Ki
    memory: 3884532Ki
    pods: "110"
  conditions:
  - lastHeartbeatTime: "2022-08-02T16:35:26Z"
    lastTransitionTime: "2022-08-02T16:33:39Z"
    message: kubelet has sufficient memory available
    reason: KubeletHasSufficientMemory
    status: "False"
    type: MemoryPressure
  - lastHeartbeatTime: "2022-08-02T16:35:26Z"
    lastTransitionTime: "2022-08-02T16:33:39Z"
    message: kubelet has no disk pressure
    reason: KubeletHasNoDiskPressure
    status: "False"
    type: DiskPressure
  - lastHeartbeatTime: "2022-08-02T16:35:26Z"
    lastTransitionTime: "2022-08-02T16:33:39Z"
    message: kubelet has sufficient PID available
    reason: KubeletHasSufficientPID
    status: "False"
    type: PIDPressure
  - lastHeartbeatTime: "2022-08-02T16:35:26Z"
    lastTransitionTime: "2022-08-02T16:35:26Z"
    message: kubelet is posting ready status
    reason: KubeletReady
    status: "True"
    type: Ready
  daemonEndpoints:
    kubeletEndpoint:
      Port: 10250
  nodeInfo:
    architecture: arm64
    bootID: 2a283cdf-2bcf-4d99-ab1e-7409ad10ae20
    containerRuntimeVersion: containerd://1.6.6-k3s1
    kernelVersion: 5.15.32-v8+
    kubeProxyVersion: v1.24.3+k3s1
    kubeletVersion: v1.24.3+k3s1
    machineID: a4565abd379244c89502a5fed73435be
    operatingSystem: linux
    osImage: Debian GNU/Linux 11 (bullseye)
    systemUUID: a4565abd379244c89502a5fed73435be

after the `--disable-network-policy` flag

kubectl get node mercury -o yaml

❯ kubectl get node mercury -o yaml
apiVersion: v1
kind: Node
metadata:
  annotations:
    alpha.kubernetes.io/provided-node-ip: xxx.xxx.xxx.xxx
    flannel.alpha.coreos.com/backend-data: '{"VNI":1,"VtepMAC":"ea:56:bb:04:9d:71"}'
    flannel.alpha.coreos.com/backend-type: vxlan
    flannel.alpha.coreos.com/backend-v6-data: '{"VNI":1,"VtepMAC":"f2:3b:21:e2:06:4f"}'
    flannel.alpha.coreos.com/kube-subnet-manager: "true"
    flannel.alpha.coreos.com/public-ip: xxx.xxx.xxx.xxx
    flannel.alpha.coreos.com/public-ipv6: xxxx:xxxx:xxxx::xxxx:xxxx
    k3s.io/hostname: mercury
    k3s.io/internal-ip: xxx.xxx.xxx.xxx,xxxx:xxxx:xxxx::xxxx:xxxx
    k3s.io/node-args: '["agent","--node-ip","xxx.xxx.xxx.xxx,xxxx:xxxx:xxxx::xxxx:xxxx","--flannel-iface","wg0","--debug"]'
    k3s.io/node-config-hash: WAGOFD7X3Q77AWWJ74YKEYBKC2DYX72ACENGVZLJDSCBUJIRTR5A====
    k3s.io/node-env: '{"K3S_DATA_DIR":"/var/lib/rancher/k3s/data/230cb7b95d48672d3d564dfde9c7b6230bccc82b2a3298f565dc3cca2a06b2eb","K3S_TOKEN":"********","K3S_URL":"https://xxx.xxx.xxx.xxx:6443"}'
    node.alpha.kubernetes.io/ttl: "0"
    volumes.kubernetes.io/controller-managed-attach-detach: "true"
  creationTimestamp: "2022-08-02T17:12:02Z"
  finalizers:
  - wrangler.cattle.io/node
  - wrangler.cattle.io/managed-etcd-controller
  labels:
    beta.kubernetes.io/arch: arm64
    beta.kubernetes.io/instance-type: k3s
    beta.kubernetes.io/os: linux
    egress.k3s.io/cluster: "true"
    kubernetes.io/arch: arm64
    kubernetes.io/hostname: mercury
    kubernetes.io/os: linux
    node.kubernetes.io/instance-type: k3s
  name: mercury
  resourceVersion: "9480"
  uid: d18a2fe4-e8c1-4178-b1f8-ec1e1c8f2efe
spec:
  podCIDR: 10.42.3.0/24
  podCIDRs:
  - 10.42.3.0/24
  - fd42:0:0:3::/64
  providerID: k3s://mercury
status:
  addresses:
  - address: xxx.xxx.xxx.xxx
    type: InternalIP
  - address: mercury
    type: Hostname
  allocatable:
    cpu: "4"
    ephemeral-storage: "467417115077"
    memory: 3884532Ki
    pods: "110"
  capacity:
    cpu: "4"
    ephemeral-storage: 480486344Ki
    memory: 3884532Ki
    pods: "110"
  conditions:
  - lastHeartbeatTime: "2022-08-02T17:32:42Z"
    lastTransitionTime: "2022-08-02T17:12:02Z"
    message: kubelet has sufficient memory available
    reason: KubeletHasSufficientMemory
    status: "False"
    type: MemoryPressure
  - lastHeartbeatTime: "2022-08-02T17:32:42Z"
    lastTransitionTime: "2022-08-02T17:12:02Z"
    message: kubelet has no disk pressure
    reason: KubeletHasNoDiskPressure
    status: "False"
    type: DiskPressure
  - lastHeartbeatTime: "2022-08-02T17:32:42Z"
    lastTransitionTime: "2022-08-02T17:12:02Z"
    message: kubelet has sufficient PID available
    reason: KubeletHasSufficientPID
    status: "False"
    type: PIDPressure
  - lastHeartbeatTime: "2022-08-02T17:32:42Z"
    lastTransitionTime: "2022-08-02T17:12:12Z"
    message: kubelet is posting ready status
    reason: KubeletReady
    status: "True"
    type: Ready
  daemonEndpoints:
    kubeletEndpoint:
      Port: 10250
  nodeInfo:
    architecture: arm64
    bootID: 2a283cdf-2bcf-4d99-ab1e-7409ad10ae20
    containerRuntimeVersion: containerd://1.6.6-k3s1
    kernelVersion: 5.15.32-v8+
    kubeProxyVersion: v1.24.3+k3s1
    kubeletVersion: v1.24.3+k3s1
    machineID: a4565abd379244c89502a5fed73435be
    operatingSystem: linux
    osImage: Debian GNU/Linux 11 (bullseye)
    systemUUID: a4565abd379244c89502a5fed73435be

Are there now any disadvantages of using the flag?
And why I have to use this, I thought my configuration was correctly or is there a bug in k3s when using a dual-stack setup?

brandond Aug 4, 2022
Collaborator

The disadvantage of doing that is that you won't have any enforcement of Kubernetes network policies. If you're not using them then you probably won't miss it.

I suspect that this regression was introduced when we updated the kube-router network policy engine to support dual-stack.

I do note that your node appears to have only a single ipv4 address listed in its status, despite having multiple node-ip addresses passed on the command line:

status:
  addresses:
  - address: xxx.xxx.xxx.xxx
    type: InternalIP
  - address: mercury
    type: Hostname

Did you by any chance start this cluster or node as single-stack ipv4, and convert it to dual-stack after the fact? If you do that, you probably need to use kubectl delete node to delete the node, and then rejoin it to the cluster, so that the addresses can be set properly.

lks-hrsch Aug 5, 2022
Author

No, I don't convert the cluster from single-stack ipv4 to dual-stack.
I always run my Ansible script to initialize a completely new cluster because there isn't much workload on it jet.

brandond Aug 5, 2022
Collaborator

Hmm. I can't reproduce this on my side. Can you share steps on how to reproduce this reliably on a new cluster install?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

level=fatal msg="host IPv6 unknown" #5922

{{title}}

Replies: 1 comment 4 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

level=fatal msg="host IPv6 unknown" #5922

lks-hrsch Jul 31, 2022

About my Cluster

About the worker mercury (Raspberry Pi 4)

About the worker earth (Raspberry Pi 3)

How I setup the cluster

Setup server

Setup agent

Steps I tried out

Replies: 1 comment · 4 replies

brandond Aug 1, 2022 Collaborator

lks-hrsch Aug 4, 2022 Author

before the --disable-network-policy flag

after the --disable-network-policy flag

brandond Aug 4, 2022 Collaborator

lks-hrsch Aug 5, 2022 Author

brandond Aug 5, 2022 Collaborator

lks-hrsch
Jul 31, 2022

Replies: 1 comment 4 replies

brandond
Aug 1, 2022
Collaborator

lks-hrsch Aug 4, 2022
Author

before the `--disable-network-policy` flag

after the `--disable-network-policy` flag

brandond Aug 4, 2022
Collaborator

lks-hrsch Aug 5, 2022
Author

brandond Aug 5, 2022
Collaborator