Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: K8sServicesTest Checks service across nodes Tests NodePort (kube-proxy) #10231

Closed
nebril opened this issue Feb 18, 2020 · 4 comments · Fixed by #12248
Closed

CI: K8sServicesTest Checks service across nodes Tests NodePort (kube-proxy) #10231

nebril opened this issue Feb 18, 2020 · 4 comments · Fixed by #12248
Assignees
@borkmann
Labels
area/CI Continuous Integration testing issue or flake
Projects
1.8 CI CI Force

Comments

@nebril
Copy link
Member

nebril commented Feb 18, 2020

CI failure

/home/jenkins/workspace/Cilium-PR-Ginkgo-Tests-K8s/1.15-gopath/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:430
Pod "testclient-6hhm2" can not connect to service "tftp://[::ffff:192.168.36.11]:30880/hello"
Expected command: kubectl exec -n default testclient-6hhm2 -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 8 tftp://[::ffff:192.168.36.11]:30880/hello -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Stdout:
 	 time-> DNS: '0.000018()', Connect: '0.000035',Transfer '0.000000', total '5.001616'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Cilium-PR-Ginkgo-Tests-K8s/1.15-gopath/src/github.com/cilium/cilium/test/k8sT/Services.go:487

Standard Output

Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Cilium pods: [cilium-9kx2j cilium-xrglx]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                          Ingress   Egress
coredns-86cd554449-49xpk     false     false
test-k8s2-5dc84ddf57-kzk6b   false     false
testclient-6hhm2             false     false
testclient-dmgkb             false     false
testds-hbx9n                 false     false
testds-lmmx9                 false     false
Cilium agent 'cilium-9kx2j': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 19 Failed 0
Cilium agent 'cilium-xrglx': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 26 Failed 0

https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-K8s/2867/

test_results_Cilium-PR-Ginkgo-Tests-K8s_2867_BDD-Test-k8s-1.15.zip
@nebril nebril added the area/CI Continuous Integration testing issue or flake label Feb 18, 2020
@borkmann
Copy link
Member

Another instance of this flake seen in master CI ginko tests:

Stacktrace
/home/jenkins/workspace/Ginkgo-CI-Tests-Pipeline/k8s-1.11-gopath/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:430
Pod "testclient-zcpgg" can not connect to service "tftp://[::ffff:192.168.36.11]:30875/hello"
Expected command: kubectl exec -n default testclient-zcpgg -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 8 tftp://[::ffff:192.168.36.11]:30875/hello -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Stdout:
 	 time-> DNS: '0.000022()', Connect: '0.000045',Transfer '0.000000', total '4.207263'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Ginkgo-CI-Tests-Pipeline/k8s-1.11-gopath/src/github.com/cilium/cilium/test/k8sT/Services.go:487
Standard Output
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Cilium pods: [cilium-lvmkb cilium-nvnn7]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                         Ingress   Egress
test-k8s2-8d47f6c76-j6vsb   false     false
testclient-zcpgg            false     false
testclient-zp9fm            false     false
testds-d9ltw                false     false
app2-dc85b4585-2j8sw        false     false
coredns-687db6485c-sxg24    false     false
testds-8r4qx                false     false
app1-798d4f944d-2wcz9       false     false
app3-68fb594d47-5zpdr       false     false
Cilium agent 'cilium-lvmkb': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 23 Failed 0
Cilium agent 'cilium-nvnn7': Status: Ok  Health: Ok Nodes "" ContinerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 26 Failed 0

Standard Error
STEP: Installing Cilium
STEP: Installing DNS Deployment
STEP: Performing Cilium preflight check
STEP: Performing Cilium status preflight check
STEP: Performing Cilium controllers preflight check
STEP: Performing Cilium health check
STEP: Performing Cilium service preflight check
STEP: Performing K8s service preflight check
STEP: Waiting for cilium-operator to be ready
STEP: Waiting for kube-dns to be ready
STEP: Running kube-dns preflight check
STEP: Performing K8s service preflight check
STEP: Making ten curl requests from "testclient-zcpgg" to "http://10.109.153.10:10080"
STEP: Making ten curl requests from "testclient-zp9fm" to "http://10.109.153.10:10080"
STEP: Making ten curl requests from "testclient-zcpgg" to "tftp://10.109.153.10:10069/hello"
STEP: Making ten curl requests from "testclient-zp9fm" to "tftp://10.109.153.10:10069/hello"
STEP: Making 10 curl requests from k8s1 to "http://127.0.0.1:32416"
STEP: Making 10 curl requests from k8s1 to "tftp://127.0.0.1:30875/hello"
STEP: Making 10 curl requests from k8s1 to "http://[::ffff:127.0.0.1]:32416"
STEP: Making 10 curl requests from k8s1 to "tftp://[::ffff:127.0.0.1]:30875/hello"
STEP: Making 10 curl requests from k8s1 to "http://192.168.36.11:32416"
STEP: Making 10 curl requests from k8s1 to "tftp://192.168.36.11:30875/hello"
STEP: Making 10 curl requests from k8s1 to "http://[::ffff:192.168.36.11]:32416"
STEP: Making 10 curl requests from k8s1 to "tftp://[::ffff:192.168.36.11]:30875/hello"
STEP: Making 10 curl requests from k8s1 to "http://192.168.36.12:32416"
STEP: Making 10 curl requests from k8s1 to "tftp://192.168.36.12:30875/hello"
STEP: Making 10 curl requests from k8s1 to "http://[::ffff:192.168.36.12]:32416"
STEP: Making 10 curl requests from k8s1 to "tftp://[::ffff:192.168.36.12]:30875/hello"
STEP: Making ten curl requests from "testclient-zcpgg" to "tftp://192.168.36.11:30875/hello"
STEP: Making ten curl requests from "testclient-zp9fm" to "tftp://192.168.36.11:30875/hello"
STEP: Making ten curl requests from "testclient-zcpgg" to "http://192.168.36.11:32416"
STEP: Making ten curl requests from "testclient-zp9fm" to "http://192.168.36.11:32416"
STEP: Making ten curl requests from "testclient-zcpgg" to "tftp://[::ffff:192.168.36.11]:30875/hello"
=== Test Finished at 2020-02-18T03:30:21Z====
===================== TEST FAILED =====================
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0 
Stdout:
 	 NAMESPACE           NAME                               READY   STATUS    RESTARTS   AGE   IP              NODE   NOMINATED NODE
	 default             test-k8s2-8d47f6c76-j6vsb          2/2     Running   0          5m    10.10.1.11      k8s2   <none>
	 default             testclient-zcpgg                   1/1     Running   0          5m    10.10.0.208     k8s1   <none>
	 default             testclient-zp9fm                   1/1     Running   0          5m    10.10.1.44      k8s2   <none>
	 default             testds-8r4qx                       2/2     Running   0          5m    10.10.0.211     k8s1   <none>
	 default             testds-d9ltw                       2/2     Running   0          5m    10.10.1.22      k8s2   <none>
	 external-ips-test   app1-798d4f944d-2wcz9              2/2     Running   0          1h    10.10.0.47      k8s1   <none>
	 external-ips-test   app2-dc85b4585-2j8sw               2/2     Running   0          1h    10.10.0.241     k8s1   <none>
	 external-ips-test   app3-68fb594d47-5zpdr              2/2     Running   0          1h    10.10.1.139     k8s2   <none>
	 external-ips-test   host-client-286nq                  1/1     Running   0          1h    192.168.36.12   k8s2   <none>
	 external-ips-test   host-client-fqjtl                  1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 external-ips-test   host-server-1-56c9467d4b-fsxvx     2/2     Running   0          1h    192.168.36.11   k8s1   <none>
	 external-ips-test   host-server-2-b8d89c58c-j2bsz      2/2     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         cilium-lvmkb                       1/1     Running   0          1m    192.168.36.12   k8s2   <none>
	 kube-system         cilium-nvnn7                       1/1     Running   0          1m    192.168.36.11   k8s1   <none>
	 kube-system         cilium-operator-54b98bf7c6-x8gw7   1/1     Running   0          32m   192.168.36.12   k8s2   <none>
	 kube-system         coredns-687db6485c-sxg24           1/1     Running   0          1h    10.10.0.205     k8s1   <none>
	 kube-system         etcd-k8s1                          1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         kube-apiserver-k8s1                1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         kube-controller-manager-k8s1       1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         kube-scheduler-k8s1                1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         log-gatherer-5xrb2                 1/1     Running   0          1h    192.168.36.13   k8s3   <none>
	 kube-system         log-gatherer-qlnf2                 1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         log-gatherer-xlnh8                 1/1     Running   0          1h    192.168.36.12   k8s2   <none>
	 kube-system         registry-adder-5m6vl               1/1     Running   0          1h    192.168.36.11   k8s1   <none>
	 kube-system         registry-adder-7svw6               1/1     Running   0          1h    192.168.36.12   k8s2   <none>
	 kube-system         registry-adder-cc8q9               1/1     Running   0          1h    192.168.36.13   k8s3   <none>
	 
Stderr:
 	 

Fetching command output from pods [cilium-lvmkb cilium-nvnn7]
cmd: kubectl exec -n kube-system cilium-lvmkb -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend               Service Type   Backend                    
	 1    10.96.0.10:53          ClusterIP      1 => 10.10.0.205:53        
	 2    10.96.0.1:443          ClusterIP      1 => 192.168.36.11:6443    
	 3    10.111.196.148:82      ClusterIP      1 => 10.10.0.47:80         
	 4    192.0.2.223:82         ExternalIPs    1 => 10.10.0.47:80         
	 5    172.28.128.3:82        ExternalIPs    1 => 10.10.0.47:80         
	 6    192.168.36.11:82       ExternalIPs    1 => 10.10.0.47:80         
	 7    10.104.137.97:30002    ClusterIP      1 => 10.10.0.47:80         
	 8    192.0.2.223:30002      ExternalIPs    1 => 10.10.0.47:80         
	 9    172.28.128.3:30002     ExternalIPs    1 => 10.10.0.47:80         
	 10   192.168.36.11:30002    ExternalIPs    1 => 10.10.0.47:80         
	 11   10.109.189.21:83       ClusterIP      1 => 10.10.0.241:80        
	 12   0.0.0.0:30003          NodePort       1 => 10.10.0.241:80        
	 14   10.10.1.97:30003       NodePort       1 => 10.10.0.241:80        
	 15   10.108.226.85:84       ClusterIP      1 => 192.168.36.11:20004   
	 16   0.0.0.0:30004          NodePort       1 => 192.168.36.11:20004   
	 18   10.10.1.97:30004       NodePort       1 => 192.168.36.11:20004   
	 19   10.104.52.41:85        ClusterIP      1 => 192.168.36.11:30006   
	 20   0.0.0.0:30005          NodePort       1 => 192.168.36.11:30006   
	 22   10.10.1.97:30005       NodePort       1 => 192.168.36.11:30006   
	 36   10.104.235.85:80       ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 37   10.104.235.85:69       ClusterIP      1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 38   10.109.153.10:10080    ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 39   10.109.153.10:10069    ClusterIP      1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 40   0.0.0.0:32416          NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 41   192.168.36.12:32416    NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 42   10.10.1.97:32416       NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 43   0.0.0.0:30875          NodePort       1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 44   192.168.36.12:30875    NodePort       1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 45   10.10.1.97:30875       NodePort       1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 46   10.110.196.85:10080    ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 47   10.110.196.85:10069    ClusterIP      1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 48   0.0.0.0:31639          NodePort       1 => 10.10.1.22:80         
	 49   192.168.36.12:31639    NodePort       1 => 10.10.1.22:80         
	 50   10.10.1.97:31639       NodePort       1 => 10.10.1.22:80         
	 51   0.0.0.0:30263          NodePort       1 => 10.10.1.22:69         
	 52   192.168.36.12:30263    NodePort       1 => 10.10.1.22:69         
	 53   10.10.1.97:30263       NodePort       1 => 10.10.1.22:69         
	 54   10.108.213.206:10080   ClusterIP      1 => 10.10.1.11:80         
	 55   10.108.213.206:10069   ClusterIP      1 => 10.10.1.11:69         
	 56   192.168.36.12:30281    NodePort       1 => 10.10.1.11:80         
	 57   10.10.1.97:30281       NodePort       1 => 10.10.1.11:80         
	 58   0.0.0.0:30281          NodePort       1 => 10.10.1.11:80         
	 59   0.0.0.0:31695          NodePort       1 => 10.10.1.11:69         
	 60   192.168.36.12:31695    NodePort       1 => 10.10.1.11:69         
	 61   10.10.1.97:31695       NodePort       1 => 10.10.1.11:69         
	 62   10.106.66.126:2379     ClusterIP                                 
	 63   10.97.146.255:10080    ClusterIP      1 => 10.10.1.11:80         
	 64   10.97.146.255:10069    ClusterIP      1 => 10.10.1.11:69         
	 65   0.0.0.0:31954          NodePort       1 => 10.10.1.11:80         
	 66   192.168.36.12:31954    NodePort       1 => 10.10.1.11:80         
	 67   10.10.1.97:31954       NodePort       1 => 10.10.1.11:80         
	 68   0.0.0.0:31048          NodePort       1 => 10.10.1.11:69         
	 69   192.168.36.12:31048    NodePort       1 => 10.10.1.11:69         
	 70   192.168.36.12:30003    NodePort       1 => 10.10.0.241:80        
	 71   192.168.36.12:30004    NodePort       1 => 192.168.36.11:20004   
	 72   10.10.1.97:31048       NodePort       1 => 10.10.1.11:69         
	 73   10.111.223.30:80       ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 74   192.168.36.12:30005    NodePort       1 => 192.168.36.11:30006   
	 75   192.168.36.12:31981    NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 76   10.10.1.97:31981       NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 77   0.0.0.0:31981          NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 78   10.108.159.24:80       ClusterIP      1 => 10.10.1.11:80         
	 79   0.0.0.0:32305          NodePort       1 => 10.10.1.11:80         
	 80   192.168.36.12:32305    NodePort       1 => 10.10.1.11:80         
	 81   10.10.1.97:32305       NodePort       1 => 10.10.1.11:80         
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-lvmkb -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                         IPv6                 IPv4          STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                         
	 1351       Disabled           Disabled          55595      k8s:io.cilium.k8s.policy.cluster=default            f00d::a0c:0:0:c164   10.10.1.11    ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=test-k8s2                                                                           
	 1565       Disabled           Disabled          5389       k8s:io.cilium.k8s.policy.cluster=default            f00d::a0c:0:0:77ab   10.10.1.44    ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=testDSClient                                                                        
	 2386       Disabled           Disabled          46042      k8s:id=app3                                         f00d::a0c:0:0:c567   10.10.1.139   ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                       
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=external-ips-test                                              
	 2481       Disabled           Disabled          17745      k8s:io.cilium.k8s.policy.cluster=default            f00d::a0c:0:0:da4    10.10.1.22    ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=testDS                                                                              
	 3975       Disabled           Disabled          4          reserved:health                                     f00d::a0c:0:0:dfa8   10.10.1.19    ready   
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-nvnn7 -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend               Service Type   Backend                    
	 1    10.96.0.1:443          ClusterIP      1 => 192.168.36.11:6443    
	 2    10.96.0.10:53          ClusterIP      1 => 10.10.0.205:53        
	 3    10.111.196.148:82      ClusterIP      1 => 10.10.0.47:80         
	 4    192.0.2.223:82         ExternalIPs    1 => 10.10.0.47:80         
	 5    172.28.128.3:82        ExternalIPs    1 => 10.10.0.47:80         
	 6    192.168.36.11:82       ExternalIPs    1 => 10.10.0.47:80         
	 7    10.104.137.97:30002    ClusterIP      1 => 10.10.0.47:80         
	 8    172.28.128.3:30002     ExternalIPs    1 => 10.10.0.47:80         
	 9    192.168.36.11:30002    ExternalIPs    1 => 10.10.0.47:80         
	 10   192.0.2.223:30002      ExternalIPs    1 => 10.10.0.47:80         
	 11   10.109.189.21:83       ClusterIP      1 => 10.10.0.241:80        
	 12   0.0.0.0:30003          NodePort       1 => 10.10.0.241:80        
	 14   10.10.0.14:30003       NodePort       1 => 10.10.0.241:80        
	 15   10.108.226.85:84       ClusterIP      1 => 192.168.36.11:20004   
	 16   0.0.0.0:30004          NodePort       1 => 192.168.36.11:20004   
	 18   10.10.0.14:30004       NodePort       1 => 192.168.36.11:20004   
	 19   10.104.52.41:85        ClusterIP      1 => 192.168.36.11:30006   
	 20   0.0.0.0:30005          NodePort       1 => 192.168.36.11:30006   
	 22   10.10.0.14:30005       NodePort       1 => 192.168.36.11:30006   
	 36   10.104.235.85:80       ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 37   10.104.235.85:69       ClusterIP      1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 38   10.109.153.10:10069    ClusterIP      1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 39   10.109.153.10:10080    ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 40   0.0.0.0:30875          NodePort       1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 41   192.168.36.11:30875    NodePort       1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 42   10.10.0.14:30875       NodePort       1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 43   0.0.0.0:32416          NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 44   192.168.36.11:32416    NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 45   10.10.0.14:32416       NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 46   10.110.196.85:10069    ClusterIP      1 => 10.10.1.22:69         
	                                            2 => 10.10.0.211:69        
	 47   10.110.196.85:10080    ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 48   0.0.0.0:30263          NodePort       1 => 10.10.0.211:69        
	 49   192.168.36.11:30263    NodePort       1 => 10.10.0.211:69        
	 50   10.10.0.14:30263       NodePort       1 => 10.10.0.211:69        
	 51   0.0.0.0:31639          NodePort       1 => 10.10.0.211:80        
	 52   192.168.36.11:31639    NodePort       1 => 10.10.0.211:80        
	 53   10.10.0.14:31639       NodePort       1 => 10.10.0.211:80        
	 54   10.108.213.206:10069   ClusterIP      1 => 10.10.1.11:69         
	 55   10.108.213.206:10080   ClusterIP      1 => 10.10.1.11:80         
	 56   10.10.0.14:31695       NodePort                                  
	 57   0.0.0.0:31695          NodePort                                  
	 58   192.168.36.11:31695    NodePort                                  
	 59   10.10.0.14:30281       NodePort                                  
	 60   0.0.0.0:30281          NodePort                                  
	 61   192.168.36.11:30281    NodePort                                  
	 62   10.106.66.126:2379     ClusterIP                                 
	 63   10.97.146.255:10069    ClusterIP      1 => 10.10.1.11:69         
	 64   10.97.146.255:10080    ClusterIP      1 => 10.10.1.11:80         
	 65   192.168.36.11:30003    NodePort       1 => 10.10.0.241:80        
	 66   192.168.36.11:30004    NodePort       1 => 192.168.36.11:20004   
	 67   0.0.0.0:31954          NodePort       1 => 10.10.1.11:80         
	 68   192.168.36.11:31954    NodePort       1 => 10.10.1.11:80         
	 69   10.10.0.14:31954       NodePort       1 => 10.10.1.11:80         
	 70   0.0.0.0:31048          NodePort       1 => 10.10.1.11:69         
	 71   192.168.36.11:31048    NodePort       1 => 10.10.1.11:69         
	 72   192.168.36.11:30005    NodePort       1 => 192.168.36.11:30006   
	 73   10.10.0.14:31048       NodePort       1 => 10.10.1.11:69         
	 74   10.111.223.30:80       ClusterIP      1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 75   0.0.0.0:31981          NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 76   192.168.36.11:31981    NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 77   10.10.0.14:31981       NodePort       1 => 10.10.1.22:80         
	                                            2 => 10.10.0.211:80        
	 78   10.108.159.24:80       ClusterIP      1 => 10.10.1.11:80         
	 79   0.0.0.0:32305          NodePort                                  
	 80   192.168.36.11:32305    NodePort                                  
	 81   10.10.0.14:32305       NodePort                                  
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-nvnn7 -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                         IPv6                 IPv4          STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                         
	 206        Disabled           Disabled          26748      k8s:id=app1                                         f00d::a0b:0:0:f96a   10.10.0.47    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                       
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=external-ips-test                                              
	 1602       Disabled           Disabled          19205      k8s:id=app2                                         f00d::a0b:0:0:d00    10.10.0.241   ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                       
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=external-ips-test                                              
	 1911       Disabled           Disabled          4          reserved:health                                     f00d::a0b:0:0:a9d    10.10.0.179   ready   
	 2448       Disabled           Disabled          52101      k8s:io.cilium.k8s.policy.cluster=default            f00d::a0b:0:0:23ab   10.10.0.205   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns                                                
	                                                            k8s:io.kubernetes.pod.namespace=kube-system                                                    
	                                                            k8s:k8s-app=kube-dns                                                                           
	 2537       Disabled           Disabled          5389       k8s:io.cilium.k8s.policy.cluster=default            f00d::a0b:0:0:bb40   10.10.0.208   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=testDSClient                                                                        
	 2726       Disabled           Disabled          17745      k8s:io.cilium.k8s.policy.cluster=default            f00d::a0b:0:0:6758   10.10.0.211   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                
	                                                            k8s:io.kubernetes.pod.namespace=default                                                        
	                                                            k8s:zgroup=testDS                                                                              
	 
Stderr:
 	 ```

@borkmann
Copy link
Member

a69f7ca1_K8sServicesTest_Checks_service_across_nodes_Tests_NodePort_BPF_Tests_with_direct_routing_Tests_NodePort (1).zip

@aanm
Copy link
Member

aanm commented Feb 26, 2020 •
edited

https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/17493/

test_results_Cilium-PR-Ginkgo-Tests-Validated_17493_BDD-Test-PR-K8s-1.17.zip

@borkmann borkmann moved this from In progress (1.8) to CI related in 1.9 kube-proxy removal & general dp optimization Mar 19, 2020
@borkmann borkmann mentioned this issue Mar 25, 2020
Merged
@gandro gandro mentioned this issue Mar 30, 2020
Merged
@aanm aanm added this to Needs triage in CI Failures via automation Apr 1, 2020
@aanm aanm moved this from Needs triage to Weekly in CI Failures Apr 1, 2020
@aanm aanm mentioned this issue Apr 1, 2020
Merged
@aanm aanm moved this from Weekly to Daily / Always in CI Failures Apr 1, 2020
@pchaigno pchaigno mentioned this issue Apr 3, 2020
Merged
@borkmann borkmann mentioned this issue Apr 28, 2020
Merged
@brb
Copy link
Member

brb commented May 5, 2020

On what kernel did the tests run? 4.9 or net-next?

@joestringer joestringer removed this from Daily / Always in CI Failures May 5, 2020
@b3a-dev b3a-dev added this to In Progress (Cilium) in CI Force May 8, 2020
@qmonnet qmonnet mentioned this issue May 12, 2020
Merged
@brb brb removed their assignment May 19, 2020
@jrajahalme jrajahalme mentioned this issue Jun 4, 2020
Merged
@pchaigno pchaigno mentioned this issue Jun 12, 2020
Merged
@jrajahalme jrajahalme mentioned this issue Jun 26, 2020
Merged
CI Force automation moved this from In Progress (Cilium) to Fixed / Done Jun 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@borkmann borkmann

Labels
area/CI Continuous Integration testing issue or flake
Projects
No open projects
1.8 CI
Awaiting triage
CI Force
  
Fixed / Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

iptables: Remove '--nowildcard' from socket match cilium/cilium
4 participants
@brb @borkmann @nebril @aanm