calico never rejects packets which belong to the kubernetes service without any endpoint

[mzyfree]

When I delete all the endpoint pods which bind to a kubernetes service,calico never rejects packet which belongs to the kubernetes service without any endpoint in the iptable rule "cali-FORWARD" in filter table.At the same time,kubernetes rejects the packet in iptables rule "KUBE-SERVICES" in filter table.

[root@node55 ~]# iptables -t filter -nvL KUBE-SERVICES
Chain KUBE-SERVICES (3 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     tcp  --  *      *       0.0.0.0/0            10.96.178.165        /* default/etcd-test: has no endpoints */ tcp dpt:2379 reject-with icmp-port-unreachable

But the "cali-FORWARD" just accepts the packet in filter table FORWARD chain,so the packets will not reject by the kubernetes iptable rule above.

[root@node56 ~]# iptables -t filter -nvL cali-pro-kns.kube-system
Chain cali-pro-kns.kube-system (6 references)
 pkts bytes target     prot opt in     out     source               destination
 471K   28M MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:-50oJuMfLVO3LkBk */ MARK or 0x10000
 471K   28M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* cali:ztVPKv1UYejNzm1g */ mark match 0x10000/0x10000

Then if anther pod wants to connects the service,it will create a nf_conntrack without any DNAT.This action may leads a problem because the wrong nf_conntrack will be alive for 2 minutes.If the pod retries to connect within 2 minutes,the wrong nf_conntrack will refresh the alive time for another 2 minutes even if the endpoint pods come back.

ipv4     2 tcp      6 118 SYN_SENT src=177.177.220.195 dst=10.96.178.165 sport=23344 dport=2379 [UNREPLIED] src=10.96.178.165 dst=100.6.5.56 sport=2379 dport=2334
4 mark=0 secctx=system_u:object_r:unlabeled_t:s0 zone=0 use=3

Expected Behavior

calico rejects packets which belong to the kubernetes service without any endpoint in "cali-FORWARD" chain in filter table.

Current Behavior

calico never rejects packets which belong to the kubernetes service without any endpoint in "cali-FORWARD" chain in filter table.

Possible Solution

Steps to Reproduce (for bugs)

1.deploy kubernetes with calico as its CNI
2.create a kubernetes service which binds to some healthy endpoint pods
3.delete all the endpoint pods
4.try to connect the kubernetes service in a client pod.

Context

Your Environment

• Calico version
  v3.7.4
• Orchestrator version (e.g. kubernetes, mesos, rkt):
  kubernetes v1.15
• Operating System and version:
  CentOS 7.5
• Link to your project (optional):

[caseydavenport]

I think this is a duplicate of the following issues (and a couple others I don't have links handy for):

• Connections to Kubernetes Services with no endpoints hang #1055
• Connections to service ClusterIP with no endpoints are not rejected kubernetes/kubernetes#78994

It should be fixed by this PR: projectcalico/felix#2424

Which, will be included in the upcoming Calico v3.16.0 release.