Redhat 8 install kubernetes, that can't communication between the podip #97283

readerx · 2020-12-14T10:07:58Z

What happened:
Redhat 8 install kubernetes, that can't communication between the podip

How to reproduce it (as minimally and precisely as possible):
install kubernetes in redhat 8.

# install kubelet、kubeadm、kubectl
yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
systemctl enable kubelet
systemctl start kubelet

# set firewall
firewall-cmd --zone=public --permanent --add-port={6443,2379,2380,10250,10251,10252}/tcp

# install cluster
kubeadm init   --pod-network-cidr=10.254.0.0/16 --image-repository registry.aliyuncs.com/google_containers

# to start using cluster
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# install calico
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

# use master node
kubectl taint nodes --all node-role.kubernetes.io/master-

podA (busybox): 10.254.219.69
podB (nginx): 10.254.219.68，listen 80 tcp port

kubectl exec -it podA -- sh

/ # wget 10.254.219.68:80
Connecting to 10.254.219.68:80 (10.254.219.68:80)
wget: can't connect to remote host (10.254.219.68): No route to host

Anything else we need to know?:

Environment:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:59:43Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.0", GitCommit:"af46c47ce925f4c4ad5cc8d1fca46c7b77d13b38", GitTreeState:"clean", BuildDate:"2020-12-08T17:51:19Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider or hardware configuration:
OS (e.g: cat /etc/os-release):

NAME="Red Hat Enterprise Linux"
VERSION="8.2 (Ootpa)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="8.2"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Red Hat Enterprise Linux 8.2 (Ootpa)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:8.2:GA"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_BUGZILLA_PRODUCT_VERSION=8.2
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="8.2"

Kernel (e.g. uname -a):

Linux localhost.localdomain 4.18.0-193.el8.x86_64 #1 SMP Fri Mar 27 14:35:58 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Install tools:
Network plugin and version (if this is a network-related bug):

calico

Others:
The problem was that the firewalld dropped the packet forwarding packet.

like this:

[root@localhost ~]# nft list chain inet firewalld filter_FORWARD
table inet firewalld {
        chain filter_FORWARD {
                type filter hook forward priority filter + 10; policy accept;
                ct state { established, related } accept
                ct status dnat accept
                iifname "lo" accept
                ip6 daddr { ::/96} reject with icmpv6 type addr-unreachable
                jump filter_FORWARD_IN_ZONES_SOURCE
                jump filter_FORWARD_IN_ZONES
                jump filter_FORWARD_OUT_ZONES_SOURCE
                jump filter_FORWARD_OUT_ZONES
                ct state { invalid } drop
                reject with icmpx type admin-prohibited
        }
}

The text was updated successfully, but these errors were encountered:

readerx · 2020-12-14T10:25:27Z

/sig network

sftim · 2020-12-14T13:24:18Z

This looks like a request for advice. The steps to reproduce: “install kubernetes in redhat 8” might not be detailed enough to replicate the issue, even knowing that Calico was also used.

/kind support

readerx · 2020-12-15T03:47:04Z

This looks like a request for advice. The steps to reproduce: “install kubernetes in redhat 8” might not be detailed enough to replicate the issue, even knowing that Calico was also used.

/kind support

tks，I added the installation process and update the comment

dougsland · 2020-12-21T20:13:11Z

This looks like a request for advice. The steps to reproduce: “install kubernetes in redhat 8” might not be detailed enough to replicate the issue, even knowing that Calico was also used.
/kind support

tks，I added the installation process and update the comment

@readerx Looks like you solved your issue report. If yes, could you please clarify it was firewalld settings? Could you please share what you did to solve it and close the issue report?

readerx · 2020-12-22T01:12:16Z

This looks like a request for advice. The steps to reproduce: “install kubernetes in redhat 8” might not be detailed enough to replicate the issue, even knowing that Calico was also used.
/kind support

tks，I added the installation process and update the comment

@readerx Looks like you solved your issue report. If yes, could you please clarify it was firewalld settings? Could you please share what you did to solve it and close the issue report?

yes, I have a solution. like this:

nft add rule inet firewalld filter_FWDO_public_allow ct state state new,untracked accept

But, I don't know if this is a good way. Whether K8S or Calico can automatically handle this firewalld settings

dougsland · 2020-12-22T14:58:58Z

Hey @readerx,

In a fresh env, what happens if you do the following:

 $ wget https://docs.projectcalico.org/manifests/calico.yaml

Add the following configuration into the env section:

- name: FELIX_IPTABLESBACKEND
  value: "NFT"

Save and apply/deploy the plugin:

$ kubectl apply -f ./calico.yaml

See also: projectcalico/calico#2322
If the above approach works, try to deploy your env via kubespray. It should handle that automatically.

dougsland · 2020-12-22T15:13:09Z

Adding @champtar into the loop. Have you seen this one in your recent tests?

sftim · 2020-12-22T15:22:10Z

I'm not convinced that this is a documentation issue for Kubernetes. Maybe this'd work better as a Calico issue or support request?

champtar · 2020-12-22T15:49:23Z

kubespray recommend to just stop firewalld, or to manage it manualy
I personally leave firewalld enabled with basically the following rules

firewall-cmd --permanent --new-zone=k8s-cluster
firewall-cmd --permanent --new-ipset=k8s-cluster --type=hash:net
firewall-cmd --permanent --zone=k8s-cluster --set-target=ACCEPT
firewall-cmd --permanent --zone=k8s-cluster --add-source=ipset:k8s-cluster
firewall-cmd --permanent --ipset=k8s-cluster --add-entry= --add-entry= ...
firewall-cmd --reload

In the add-entry I put the nodes ips, the clusterIPs and podIPs cidr
I have no idea if it affect Calico network policies as I don't use them, but for sure your issue here is firewalld conf

your nft rule just allow any forwarding, might be too permissive

dougsland · 2020-12-22T16:04:01Z

I'm not convinced that this is a documentation issue for Kubernetes. Maybe this'd work better as a Calico issue or support
request?

Agreed and I believe the trick I have mentioned in #97283 (comment) should make it work.

champtar · 2020-12-22T16:34:34Z

@dougsland right, you also definitly need to set FELIX_IPTABLESBACKEND to NFT or Auto, else you have mixed iptables-nft and iptables-legacy usage, which is undefined.
@readerx Fix the config and reboot everything

readerx · 2020-12-23T02:27:19Z

@dougsland right, you also definitly need to set FELIX_IPTABLESBACKEND to NFT or Auto, else you have mixed iptables-nft and iptables-legacy usage, which is undefined.
@readerx Fix the config and reboot everything

I used this configuration in calico-node

        - name: FELIX_IPTABLESBACKEND
          value: NFT

nft ruleset like this

[root@master ~]# nft list table ip filter
table ip filter {
        chain INPUT {
                type filter hook input priority filter; policy accept;
                 counter packets 65069 bytes 11110763 jump cali-INPUT
                ct state new  counter packets 1405 bytes 99965 jump KUBE-EXTERNAL-SERVICES
                counter packets 89884 bytes 21152784 jump KUBE-FIREWALL
        }

        chain FORWARD {
                type filter hook forward priority filter; policy drop;
                 counter packets 1914 bytes 86557 jump cali-FORWARD
                 counter packets 1961 bytes 89356 jump KUBE-FORWARD
                ct state new  counter packets 1961 bytes 89356 jump KUBE-SERVICES
                ct state new  counter packets 1961 bytes 89356 jump KUBE-EXTERNAL-SERVICES
                counter packets 1961 bytes 89356 jump DOCKER-USER
                counter packets 1961 bytes 89356 jump DOCKER-ISOLATION-STAGE-1
                oifname "docker0" ct state related,established counter packets 0 bytes 0 accept
                oifname "docker0" counter packets 0 bytes 0 jump DOCKER
                iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 accept
                iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
                  mark and 0x10000 == 0x10000 counter packets 1914 bytes 86557 accept
        }

        chain OUTPUT {
                type filter hook output priority filter; policy accept;
                 counter packets 65272 bytes 11215242 jump cali-OUTPUT
                ct state new  counter packets 1152 bytes 68754 jump KUBE-SERVICES
                counter packets 90976 bytes 21414312 jump KUBE-FIREWALL
        }

        chain DOCKER {
        }

        chain DOCKER-ISOLATION-STAGE-1 {
                iifname "docker0" oifname != "docker0" counter packets 0 bytes 0 jump DOCKER-ISOLATION-STAGE-2
                counter packets 1961 bytes 89356 return
        }

        chain DOCKER-ISOLATION-STAGE-2 {
                oifname "docker0" counter packets 0 bytes 0 drop
                counter packets 0 bytes 0 return
        }

        chain DOCKER-USER {
                counter packets 1961 bytes 89356 return
        }

        chain KUBE-FIREWALL {
                 mark and 0x8000 == 0x8000 counter packets 0 bytes 0 drop
                ip saddr != 127.0.0.0/8 ip daddr 127.0.0.0/8  ct state != related,established counter packets 0 bytes 0 drop
        }

        chain KUBE-KUBELET-CANARY {
        }

        chain KUBE-PROXY-CANARY {
        }

        chain KUBE-EXTERNAL-SERVICES {
        }

        chain KUBE-SERVICES {
        }

        chain KUBE-FORWARD {
                ct state invalid counter packets 0 bytes 0 drop
                 mark and 0x4000 == 0x4000 counter packets 0 bytes 0 accept
                 ct state related,established counter packets 0 bytes 0 accept
                 ct state related,established counter packets 0 bytes 0 accept
        }

        chain cali-to-wl-dispatch {
                oifname "cali7f5f7ec636d"  counter packets 0 bytes 0 goto cali-tw-cali7f5f7ec636d
                oifname "calib08de47b6f5"  counter packets 0 bytes 0 goto cali-tw-calib08de47b6f5
                oifname "calie*"  counter packets 2 bytes 120 goto cali-to-wl-dispatch-e
                  counter packets 0 bytes 0 drop
        }

        chain cali-to-host-endpoint {
        }

        chain cali-pro-kns.default {
                 counter packets 2 bytes 120 meta mark set mark or 0x10000
                 mark and 0x10000 == 0x10000 counter packets 2 bytes 120 return
        }

      ......

But problems remain

[root@master ~]# kubectl get po -A -o wide
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE     IP              NODE     NOMINATED NODE   READINESS GATES
default       busybox-deploy-cb87fccb5-2q45l             1/1     Running   3          7d22h   10.254.219.82   master   <none>           <none>
default       nginx-deploy-7fb7fd49b4-cz4nk              1/1     Running   3          7d22h   10.254.219.84   master   <none>           <none>

[root@master ~]# kubectl exec -it busybox-deploy-cb87fccb5-2q45l -- sh
/ # wget http://10.254.219.84
Connecting to 10.254.219.84 (10.254.219.84:80)
wget: can't connect to remote host (10.254.219.84): No route to host

I think the reason for the problem is iptabes-nft operations is ip family, but firewalld operations inet family.
firewalld filter_FORWARD chain of inet family drops the packet between the pod.

If want to solve this problem Calico needs to handle the inet firewalld table. Of course this is just my idea, I don't know if it's right.

champtar · 2020-12-23T02:39:27Z

I would first test without firewalld (disable & reboot all nodes). If it work this is not a k8s bug ;), and you can enable it again, configure it as I explained, and restart your debug journey from there.

readerx · 2020-12-23T03:11:44Z

I would first test without firewalld (disable & reboot all nodes). If it work this is not a k8s bug ;), and you can enable it again, configure it as I explained, and restart your debug journey from there.

Yes, After the firewall is turned off, the problem does not appear.
Does that make this question more appropriate for Calico? : )

caseydavenport · 2021-01-07T23:54:49Z

So, our general recommendation on the Calico side is to just disable firewalld and instead to use Calico to manage the firewall rules on the host. I think that's what most folks end up doing.

I believe that it is possible to make Calico and firewalld play nicely together, but I'm afraid I don't know the full set of configurations needed in firewalld to make this happen off the top of my head. Some things to look out for would be:

Make sure IPIP / VXLAN / BGP traffic is allowed by firewalld depending on your Calico config.
Make sure firewalld isn't performing NAT on pod traffic, which could interfere with network policies.

At the end of the day, though, I don't think this is really a Kubernetes bug or feature request, since the interactions are solely between Calico and firewalld.

/remove-kind bug
/triage accepted
/close

k8s-ci-robot · 2021-01-07T23:55:04Z

@caseydavenport: Closing this issue.

In response to this:

So, our general recommendation on the Calico side is to just disable firewalld and instead to use Calico to manage the firewall rules on the host. I think that's what most folks end up doing.

I believe that it is possible to make Calico and firewalld play nicely together, but I'm afraid I don't know the full set of configurations needed in firewalld to make this happen off the top of my head. Some things to look out for would be:

Make sure IPIP / VXLAN / BGP traffic is allowed by firewalld depending on your Calico config.

Make sure firewalld isn't performing NAT on pod traffic, which could interfere with network policies.

At the end of the day, though, I don't think this is really a Kubernetes bug or feature request, since the interactions are solely between Calico and firewalld.

/remove-kind bug
/triage accepted
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

caseydavenport · 2021-01-07T23:55:35Z

I've left the corresponding Calico issue open since we should probably document something on that side.

sagarvelankar · 2021-02-28T08:33:26Z

Hi,

I solved this problem in CentOS 8 by creating a new firewalld zone for kubernetes pods and setting its target to ACCEPT.
So, firewalld will accept packets going into POD SUBNET CIDR (ingress zone) and also packets coming out of POD SUBNET CIDR
(egress zone)

Commands :

firewall-cmd --permanent --delete-zone=kubernetes_pods
firewall-cmd --permanent --new-zone=kubernetes_pods
firewall-cmd --permanent --zone=kubernetes_pods --set-target=ACCEPT
firewall-cmd --permanent --zone=kubernetes_pods --add-source=<POD SUBNET CIDR>
firewall-cmd --reload

firewall-cmd man :

       --permanent [--zone=zone] --set-target=target
           Set the target of a permanent zone.  target is one of: default, ACCEPT, DROP, REJECT

           default is similar to REJECT, but has special meaning in the following scenarios:

            1. ICMP explicitly allowed

               At the end of the zone's ruleset ICMP packets are explicitly allowed.

            2. forwarded packets follow the target of the egress zone

               In the case of forwarded packets, if the ingress zone uses default then whether or not the packet will be
               allowed is determined by the egress zone.

               For a forwarded packet that ingresses zoneA and egresses zoneB:

               ·   if zoneA's target is ACCEPT, DROP, or REJECT then the packet is accepted, dropped, or rejected
                   respectively.

               ·   if zoneA's target is default, then the packet is accepted, dropped, or rejected based on zoneB's target. If
                   zoneB's target is also default, then the packet will be rejected by firewalld's catchall reject.

            3. Zone drifting from source-based zone to interface-based zone

               This only applies if AllowZoneDrifting is enabled. See firewalld.conf(5).

               If a packet ingresses a source-based zone with a target of default, it may still enter an interface-based zone
               (including the default zone).

Versions :

firewall-cmd --version
0.8.2

uname -r
4.18.0-240.10.1.el8_3.x86_64

cat /etc/redhat-release
CentOS Linux release 8.3.2011

kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:20:00Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

To see what is getting rejected by firewalld, use the below commands

firewall-cmd --set-log-denied=all
firewall-cmd --reload
dmesg | grep -i reject

readerx · 2021-03-02T02:58:16Z

Hi,

I solved this problem in CentOS 8 by creating a new firewalld zone for kubernetes pods and setting its target to ACCEPT.
So, firewalld will accept packets going into POD SUBNET CIDR (ingress zone) and also packets coming out of POD SUBNET CIDR
(egress zone)

Commands :

firewall-cmd --permanent --delete-zone=kubernetes_pods
firewall-cmd --permanent --new-zone=kubernetes_pods
firewall-cmd --permanent --zone=kubernetes_pods --set-target=ACCEPT
firewall-cmd --permanent --zone=kubernetes_pods --add-source=<POD SUBNET CIDR>
firewall-cmd --reload

firewall-cmd man :

       --permanent [--zone=zone] --set-target=target
           Set the target of a permanent zone.  target is one of: default, ACCEPT, DROP, REJECT

           default is similar to REJECT, but has special meaning in the following scenarios:

            1. ICMP explicitly allowed

               At the end of the zone's ruleset ICMP packets are explicitly allowed.

            2. forwarded packets follow the target of the egress zone

               In the case of forwarded packets, if the ingress zone uses default then whether or not the packet will be
               allowed is determined by the egress zone.

               For a forwarded packet that ingresses zoneA and egresses zoneB:

               ·   if zoneA's target is ACCEPT, DROP, or REJECT then the packet is accepted, dropped, or rejected
                   respectively.

               ·   if zoneA's target is default, then the packet is accepted, dropped, or rejected based on zoneB's target. If
                   zoneB's target is also default, then the packet will be rejected by firewalld's catchall reject.

            3. Zone drifting from source-based zone to interface-based zone

               This only applies if AllowZoneDrifting is enabled. See firewalld.conf(5).

               If a packet ingresses a source-based zone with a target of default, it may still enter an interface-based zone
               (including the default zone).

Versions :

firewall-cmd --version
0.8.2

uname -r
4.18.0-240.10.1.el8_3.x86_64

cat /etc/redhat-release
CentOS Linux release 8.3.2011

kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:28:09Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2", GitCommit:"faecb196815e248d3ecfb03c680a4507229c2a56", GitTreeState:"clean", BuildDate:"2021-01-13T13:20:00Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}

To see what is getting rejected by firewalld, use the below commands

firewall-cmd --set-log-denied=all
firewall-cmd --reload
dmesg | grep -i reject

Think you.
The cause of this problem is that the CentOS 8 Firewalld and Calico conflict.
If I don't change the Firewalld rule that's what I want.

sfgroups-k8s · 2021-03-28T01:13:31Z

Hi,
I have configured calico with FELIX_IPTABLESBACKEND as NFT, but still RHEL 8 server not able communicate PODs running on two different host.
Here is my config

Firewalld is disabled
SELINUX is disabled
Kubererntes version v1.18.6
Kernel version : 4.18.0-240.15.1.el8_3.x86_64

 image: docker.io/calico/node:v3.18.1
         envFrom:
         - configMapRef:
             # Allow KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT to be overridden for
eBPF mode.
             name: kubernetes-services-endpoint
             optional: true
         env:
           - name: FELIX_IPTABLESBACKEND
             value: NFT
           # Use Kubernetes API as the backing datastore.
           - name: DATASTORE_TYPE
             value: "kubernetes"

looks like this config worked on CentOS 8 steam, but its not working for me

https://mihail-milev.medium.com/no-pod-to-pod-communication-on-centos-8-kubernetes-with-calico-56d694d2a6f4

Any help to fix this issue?

Thanks

champtar · 2021-03-28T04:01:02Z

IPIP offload is broken with at least Mellanox and VMware in RHEL 8.3, and Calico defaults to IPIP, so have a look at this bug this might be your problem: projectcalico/calico#4384

sfgroups-k8s · 2021-03-28T17:42:46Z

Thanks based on this I need to run these two commands, My server running on VMWare:

kubernetes-sigs/kubespray#7268

ethtool -K ens192 tx-udp_tnl-csum-segmentation off
ethtool -K ens192 tx-udp_tnl-segmentation off

to keep this setting during reboot:

# cat /etc/sysconfig/network-scripts/ifcfg-ens192 | grep ETHTOOL
ETHTOOL_OPTS="-K ens192 tx-udp_tnl-csum-segmentation off; -K ens192 tx-udp_tnl-segmentation off"

champtar · 2021-03-28T18:03:57Z

Or you can switch from IPIP to VXLAN or even pure routing ;)

readerx · 2021-03-29T01:21:31Z

@sfgroups-k8s Have you solved your problem yet?

The firewall is off in your environment，So it's not the same problem that I have

readerx added the kind/bug Categorizes issue or PR as related to a bug. label Dec 14, 2020

k8s-ci-robot added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 14, 2020

k8s-ci-robot added sig/network Categorizes an issue or PR as relevant to SIG Network. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 14, 2020

k8s-ci-robot added the kind/support Categorizes issue or PR as a support question. label Dec 14, 2020

thockin assigned caseydavenport Jan 7, 2021

k8s-ci-robot added triage/accepted Indicates an issue or PR is ready to be actively worked on. and removed kind/bug Categorizes issue or PR as related to a bug. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Jan 7, 2021

k8s-ci-robot closed this as completed Jan 7, 2021

petermicuch mentioned this issue Nov 5, 2021

Reload NPC iptables rules if they get cleared weaveworks/weave#3919

Open

readerx mentioned this issue Dec 15, 2021

Document interactions with firewalld projectcalico/calico#5232

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Redhat 8 install kubernetes, that can't communication between the podip #97283

Redhat 8 install kubernetes, that can't communication between the podip #97283

readerx commented Dec 14, 2020 •

edited

readerx commented Dec 14, 2020

sftim commented Dec 14, 2020

readerx commented Dec 15, 2020

dougsland commented Dec 21, 2020 •

edited

readerx commented Dec 22, 2020

dougsland commented Dec 22, 2020 •

edited

dougsland commented Dec 22, 2020 •

edited

sftim commented Dec 22, 2020

champtar commented Dec 22, 2020

dougsland commented Dec 22, 2020

champtar commented Dec 22, 2020

readerx commented Dec 23, 2020

champtar commented Dec 23, 2020

readerx commented Dec 23, 2020

caseydavenport commented Jan 7, 2021

k8s-ci-robot commented Jan 7, 2021

caseydavenport commented Jan 7, 2021

sagarvelankar commented Feb 28, 2021

readerx commented Mar 2, 2021

sfgroups-k8s commented Mar 28, 2021

champtar commented Mar 28, 2021

sfgroups-k8s commented Mar 28, 2021

champtar commented Mar 28, 2021

readerx commented Mar 29, 2021

Redhat 8 install kubernetes, that can't communication between the podip #97283

Redhat 8 install kubernetes, that can't communication between the podip #97283

Comments

readerx commented Dec 14, 2020 • edited

readerx commented Dec 14, 2020

sftim commented Dec 14, 2020

readerx commented Dec 15, 2020

dougsland commented Dec 21, 2020 • edited

readerx commented Dec 22, 2020

dougsland commented Dec 22, 2020 • edited

dougsland commented Dec 22, 2020 • edited

sftim commented Dec 22, 2020

champtar commented Dec 22, 2020

dougsland commented Dec 22, 2020

champtar commented Dec 22, 2020

readerx commented Dec 23, 2020

champtar commented Dec 23, 2020

readerx commented Dec 23, 2020

caseydavenport commented Jan 7, 2021

k8s-ci-robot commented Jan 7, 2021

caseydavenport commented Jan 7, 2021

sagarvelankar commented Feb 28, 2021

readerx commented Mar 2, 2021

sfgroups-k8s commented Mar 28, 2021

champtar commented Mar 28, 2021

sfgroups-k8s commented Mar 28, 2021

champtar commented Mar 28, 2021

readerx commented Mar 29, 2021

readerx commented Dec 14, 2020 •

edited

dougsland commented Dec 21, 2020 •

edited

dougsland commented Dec 22, 2020 •

edited

dougsland commented Dec 22, 2020 •

edited