Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: K8sDatapathServicesTest Checks N/S loadbalancing Tests with XDP, vxlan tunnel, SNAT and Random #22579

Closed
maintainer-s-little-helper bot opened this issue Dec 6, 2022 · 8 comments
Closed

CI: K8sDatapathServicesTest Checks N/S loadbalancing Tests with XDP, vxlan tunnel, SNAT and Random #22579

maintainer-s-little-helper bot opened this issue Dec 6, 2022 · 8 comments
Labels
area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me! sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.

Comments

@maintainer-s-little-helper
Copy link

Test Name

K8sDatapathServicesTest Checks N/S loadbalancing Tests with XDP, vxlan tunnel, SNAT and Random

Failure Output

FAIL: Can not connect to service "http://192.168.56.11:32718" from outside cluster (1/10)

Stacktrace

Click to show. 
/home/jenkins/workspace/Cilium-PR-K8s-1.26-kernel-net-next/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:515
Can not connect to service "http://192.168.56.11:32718" from outside cluster (1/10)
Expected command: kubectl exec -n kube-system log-gatherer-tj9x9 -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://192.168.56.11:32718 -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000016()', Connect: '0.000000',Transfer '0.000000', total '5.001951'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Cilium-PR-K8s-1.26-kernel-net-next/src/github.com/cilium/cilium/test/k8s/service_helpers.go:253

Standard Output

Click to show. 
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Cilium pods: [cilium-7c7qn cilium-mfxmn]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                           Ingress   Egress
app3-d5f4b5fb5-5tzh8          false     false
test-k8s2-76f4bdcf49-fgsck    false     false
app1-5989697f89-8l25t         false     false
testclient-qzv5j              false     false
testds-x6kd9                  false     false
coredns-6d97d5ddb-zq6hb       false     false
testclient-4t8dq              false     false
grafana-698bbd89d5-kn2kp      false     false
app1-5989697f89-czpxw         false     false
app2-7fd4ddfc6-vsgct          false     false
echo-7945986bb5-mpkcj         false     false
prometheus-75fd6464d8-hhq2x   false     false
echo-7945986bb5-2fn6w         false     false
testds-rh5np                  false     false
Cilium agent 'cilium-7c7qn': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 64 Failed 0
Cilium agent 'cilium-mfxmn': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 44 Failed 0

Standard Error

Click to show. 
12:29:57 STEP: Installing Cilium
12:29:59 STEP: Waiting for Cilium to become ready
12:30:24 STEP: Validating if Kubernetes DNS is deployed
12:30:24 STEP: Checking if deployment is ready
12:30:24 STEP: Checking if kube-dns service is plumbed correctly
12:30:24 STEP: Checking if pods have identity
12:30:24 STEP: Checking if DNS can resolve
12:30:34 STEP: Kubernetes DNS is up and operational
12:30:34 STEP: Validating Cilium Installation
12:30:34 STEP: Performing Cilium controllers preflight check
12:30:34 STEP: Performing Cilium health check
12:30:34 STEP: Checking whether host EP regenerated
12:30:34 STEP: Performing Cilium status preflight check
12:30:36 STEP: Performing Cilium service preflight check
12:30:36 STEP: Performing K8s service preflight check
12:30:37 STEP: Waiting for cilium-operator to be ready
12:30:37 STEP: WaitforPods(namespace="kube-system", filter="-l name=cilium-operator")
12:30:37 STEP: WaitforPods(namespace="kube-system", filter="-l name=cilium-operator") => <nil>
12:30:37 STEP: Making 10 HTTP requests from outside cluster to "http://192.168.56.11:32718"
FAIL: Can not connect to service "http://192.168.56.11:32718" from outside cluster (1/10)
Expected command: kubectl exec -n kube-system log-gatherer-tj9x9 -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://192.168.56.11:32718 -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000016()', Connect: '0.000000',Transfer '0.000000', total '5.001951'
Stderr:
 	 command terminated with exit code 28
	 

=== Test Finished at 2022-12-06T12:30:42Z====
12:30:42 STEP: Running JustAfterEach block for EntireTestsuite K8sDatapathServicesTest
===================== TEST FAILED =====================
12:30:42 STEP: Running AfterFailed block for EntireTestsuite K8sDatapathServicesTest
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0 
Stdout:
 	 NAMESPACE           NAME                              READY   STATUS    RESTARTS   AGE     IP              NODE   NOMINATED NODE   READINESS GATES
	 cilium-monitoring   grafana-698bbd89d5-kn2kp          1/1     Running   0          49m     10.0.0.225      k8s1   <none>           <none>
	 cilium-monitoring   prometheus-75fd6464d8-hhq2x       1/1     Running   0          49m     10.0.0.75       k8s1   <none>           <none>
	 default             app1-5989697f89-8l25t             2/2     Running   0          7m28s   10.0.0.216      k8s1   <none>           <none>
	 default             app1-5989697f89-czpxw             2/2     Running   0          7m28s   10.0.0.141      k8s1   <none>           <none>
	 default             app2-7fd4ddfc6-vsgct              1/1     Running   0          7m28s   10.0.0.251      k8s1   <none>           <none>
	 default             app3-d5f4b5fb5-5tzh8              1/1     Running   0          7m28s   10.0.0.60       k8s1   <none>           <none>
	 default             echo-7945986bb5-2fn6w             2/2     Running   0          7m27s   10.0.0.49       k8s1   <none>           <none>
	 default             echo-7945986bb5-mpkcj             2/2     Running   0          7m27s   10.0.1.96       k8s2   <none>           <none>
	 default             test-k8s2-76f4bdcf49-fgsck        2/2     Running   0          7m28s   10.0.1.54       k8s2   <none>           <none>
	 default             testclient-4t8dq                  1/1     Running   0          7m28s   10.0.1.36       k8s2   <none>           <none>
	 default             testclient-qzv5j                  1/1     Running   0          7m28s   10.0.0.70       k8s1   <none>           <none>
	 default             testds-rh5np                      2/2     Running   0          7m19s   10.0.1.100      k8s2   <none>           <none>
	 default             testds-x6kd9                      2/2     Running   0          7m13s   10.0.0.42       k8s1   <none>           <none>
	 kube-system         cilium-7c7qn                      1/1     Running   0          45s     192.168.56.11   k8s1   <none>           <none>
	 kube-system         cilium-mfxmn                      1/1     Running   0          45s     192.168.56.12   k8s2   <none>           <none>
	 kube-system         cilium-operator-cc98f7fcf-t7zxx   1/1     Running   0          45s     192.168.56.12   k8s2   <none>           <none>
	 kube-system         cilium-operator-cc98f7fcf-twkx4   1/1     Running   0          45s     192.168.56.13   k8s3   <none>           <none>
	 kube-system         coredns-6d97d5ddb-zq6hb           1/1     Running   0          9m32s   10.0.1.242      k8s2   <none>           <none>
	 kube-system         etcd-k8s1                         1/1     Running   0          59m     192.168.56.11   k8s1   <none>           <none>
	 kube-system         kube-apiserver-k8s1               1/1     Running   0          59m     192.168.56.11   k8s1   <none>           <none>
	 kube-system         kube-controller-manager-k8s1      1/1     Running   0          59m     192.168.56.11   k8s1   <none>           <none>
	 kube-system         kube-scheduler-k8s1               1/1     Running   0          59m     192.168.56.11   k8s1   <none>           <none>
	 kube-system         log-gatherer-57x9j                1/1     Running   0          49m     192.168.56.11   k8s1   <none>           <none>
	 kube-system         log-gatherer-ftw2w                1/1     Running   0          49m     192.168.56.12   k8s2   <none>           <none>
	 kube-system         log-gatherer-tj9x9                1/1     Running   0          49m     192.168.56.13   k8s3   <none>           <none>
	 kube-system         registry-adder-hd69p              1/1     Running   0          49m     192.168.56.11   k8s1   <none>           <none>
	 kube-system         registry-adder-lwwjs              1/1     Running   0          49m     192.168.56.12   k8s2   <none>           <none>
	 kube-system         registry-adder-w4rss              1/1     Running   0          49m     192.168.56.13   k8s3   <none>           <none>
	 
Stderr:
 	 

Fetching command output from pods [cilium-7c7qn cilium-mfxmn]
cmd: kubectl exec -n kube-system cilium-7c7qn -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID    Frontend                Service Type   Backend                            
	 1     10.96.0.10:53           ClusterIP      1 => 10.0.1.242:53 (active)        
	 2     10.96.0.10:9153         ClusterIP      1 => 10.0.1.242:9153 (active)      
	 3     10.105.101.62:3000      ClusterIP      1 => 10.0.0.225:3000 (active)      
	 4     10.107.193.248:9090     ClusterIP      1 => 10.0.0.75:9090 (active)       
	 6     10.96.0.1:443           ClusterIP      1 => 192.168.56.11:6443 (active)   
	 8     10.96.250.178:80        ClusterIP      1 => 10.0.0.141:80 (active)        
	                                              2 => 10.0.0.216:80 (active)        
	 9     10.96.250.178:69        ClusterIP      1 => 10.0.0.141:69 (active)        
	                                              2 => 10.0.0.216:69 (active)        
	 10    10.103.117.146:69       ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 11    10.103.117.146:80       ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 12    10.107.51.240:10080     ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 13    10.107.51.240:10069     ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 16    192.168.56.11:32718     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 17    0.0.0.0:32718           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 19    192.168.56.11:30091     NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 20    0.0.0.0:30091           NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 22    10.106.94.23:10080      ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 23    10.106.94.23:10069      ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 26    0.0.0.0:32706           NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 27    192.168.56.11:32706     NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 29    192.168.56.11:30011     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 31    0.0.0.0:30011           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 32    10.102.222.21:10069     ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 33    10.102.222.21:10080     ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 38    192.168.56.11:30494     NodePort       1 => 10.0.0.42:80 (active)         
	 39    192.168.56.11:30494/i   NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 40    0.0.0.0:30494           NodePort       1 => 10.0.0.42:80 (active)         
	 41    0.0.0.0:30494/i         NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 46    192.168.56.11:31538     NodePort       1 => 10.0.0.42:69 (active)         
	 47    192.168.56.11:31538/i   NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 48    0.0.0.0:31538           NodePort       1 => 10.0.0.42:69 (active)         
	 49    0.0.0.0:31538/i         NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 50    10.98.99.140:10069      ClusterIP      1 => 10.0.1.54:69 (active)         
	 51    10.98.99.140:10080      ClusterIP      1 => 10.0.1.54:80 (active)         
	 52    0.0.0.0:30975           NodePort                                          
	 53    0.0.0.0:30975/i         NodePort       1 => 10.0.1.54:69 (active)         
	 58    192.168.56.11:30975     NodePort                                          
	 59    192.168.56.11:30975/i   NodePort       1 => 10.0.1.54:69 (active)         
	 60    192.168.56.11:32642     NodePort                                          
	 61    192.168.56.11:32642/i   NodePort       1 => 10.0.1.54:80 (active)         
	 62    0.0.0.0:32642           NodePort                                          
	 63    0.0.0.0:32642/i         NodePort       1 => 10.0.1.54:80 (active)         
	 68    10.105.87.113:10080     ClusterIP      1 => 10.0.1.54:80 (active)         
	 69    10.105.87.113:10069     ClusterIP      1 => 10.0.1.54:69 (active)         
	 70    192.168.56.11:32104     NodePort       1 => 10.0.1.54:80 (active)         
	 71    0.0.0.0:32104           NodePort       1 => 10.0.1.54:80 (active)         
	 74    0.0.0.0:30713           NodePort       1 => 10.0.1.54:69 (active)         
	 77    192.168.56.11:30713     NodePort       1 => 10.0.1.54:69 (active)         
	 78    10.102.106.193:80       ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 79    192.168.56.11:31902     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 80    0.0.0.0:31902           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 83    10.99.164.251:80        ClusterIP      1 => 10.0.1.54:80 (active)         
	 86    192.168.56.11:30529     NodePort                                          
	 87    192.168.56.11:30529/i   NodePort       1 => 10.0.1.54:80 (active)         
	 88    0.0.0.0:30529           NodePort                                          
	 89    0.0.0.0:30529/i         NodePort       1 => 10.0.1.54:80 (active)         
	 92    10.109.187.42:20069     ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 93    10.109.187.42:20080     ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 94    192.0.2.233:20080       ExternalIPs    1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 95    192.0.2.233:20069       ExternalIPs    1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 96    192.168.56.11:31748     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 97    0.0.0.0:31748           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 101   0.0.0.0:32629           NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 102   192.168.56.11:32629     NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 104   10.106.131.128:80       ClusterIP      1 => 10.0.1.96:80 (active)         
	                                              2 => 10.0.0.49:80 (active)         
	 105   10.106.131.128:69       ClusterIP      1 => 10.0.1.96:69 (active)         
	                                              2 => 10.0.0.49:69 (active)         
	 108   192.168.56.11:30951     NodePort       1 => 10.0.1.96:80 (active)         
	                                              2 => 10.0.0.49:80 (active)         
	 109   0.0.0.0:30951           NodePort       1 => 10.0.1.96:80 (active)         
	                                              2 => 10.0.0.49:80 (active)         
	 110   192.168.56.11:30791     NodePort       1 => 10.0.1.96:69 (active)         
	                                              2 => 10.0.0.49:69 (active)         
	 113   0.0.0.0:30791           NodePort       1 => 10.0.1.96:69 (active)         
	                                              2 => 10.0.0.49:69 (active)         
	 114   [fd03::f7d5]:80         ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 115   [fd03::f7d5]:69         ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 116   [fd03::16fe]:10069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 117   [fd03::16fe]:10080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 119   [fd04::11]:30528        NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 120   [::]:30528              NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 122   [fd04::11]:32108        NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 123   [::]:32108              NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 124   [fd03::6a2a]:10080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 125   [fd03::6a2a]:10069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 127   [fd04::11]:30386        NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 128   [::]:30386              NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 130   [fd04::11]:30583        NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 131   [::]:30583              NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 132   [fd03::b838]:10080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 133   [fd03::b838]:10069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 134   [::]:31368              NodePort       1 => [fd02::60]:80 (active)        
	 135   [::]:31368/i            NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 138   [fd04::11]:31368        NodePort       1 => [fd02::60]:80 (active)        
	 139   [fd04::11]:31368/i      NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 140   [fd04::11]:32717        NodePort       1 => [fd02::60]:69 (active)        
	 141   [fd04::11]:32717/i      NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 144   [::]:32717              NodePort       1 => [fd02::60]:69 (active)        
	 145   [::]:32717/i            NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 146   [fd03::9e59]:10080      ClusterIP      1 => [fd02::1bb]:80 (active)       
	 147   [fd03::9e59]:10069      ClusterIP      1 => [fd02::1bb]:69 (active)       
	 150   [fd04::11]:30100        NodePort                                          
	 151   [fd04::11]:30100/i      NodePort       1 => [fd02::1bb]:80 (active)       
	 152   [::]:30100              NodePort                                          
	 153   [::]:30100/i            NodePort       1 => [fd02::1bb]:80 (active)       
	 154   [::]:30316              NodePort                                          
	 155   [::]:30316/i            NodePort       1 => [fd02::1bb]:69 (active)       
	 158   [fd04::11]:30316        NodePort                                          
	 159   [fd04::11]:30316/i      NodePort       1 => [fd02::1bb]:69 (active)       
	 160   [fd03::8f4]:10080       ClusterIP      1 => [fd02::1bb]:80 (active)       
	 161   [fd03::8f4]:10069       ClusterIP      1 => [fd02::1bb]:69 (active)       
	 163   [fd04::11]:31122        NodePort       1 => [fd02::1bb]:69 (active)       
	 164   [::]:31122              NodePort       1 => [fd02::1bb]:69 (active)       
	 166   [fd04::11]:31035        NodePort       1 => [fd02::1bb]:80 (active)       
	 167   [::]:31035              NodePort       1 => [fd02::1bb]:80 (active)       
	 168   [fd03::d8f4]:20080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 169   [fd03::d8f4]:20069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 170   [fd03::999]:20080       ExternalIPs    1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 171   [fd03::999]:20069       ExternalIPs    1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 173   [fd04::11]:32381        NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 174   [::]:32381              NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 175   [::]:32356              NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 177   [fd04::11]:32356        NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 178   192.168.56.11:20069     ExternalIPs    1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 179   192.168.56.11:20080     ExternalIPs    1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 180   [fd04::11]:20080        ExternalIPs    1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 181   [fd04::11]:20069        ExternalIPs    1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 182   10.106.137.186:443      ClusterIP      1 => 192.168.56.12:4244 (active)   
	                                              2 => 192.168.56.11:4244 (active)   
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-7c7qn -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                                                        IPv6       IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                                             
	 74         Disabled           Disabled          9378       k8s:id=app1                                                                        fd02::94   10.0.0.216   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default                                             
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:zgroup=testapp                                                                                                 
	 261        Disabled           Disabled          39750      k8s:app=grafana                                                                    fd02::de   10.0.0.225   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                    
	                                                            k8s:io.kubernetes.pod.namespace=cilium-monitoring                                                                  
	 310        Disabled           Disabled          62384      k8s:appSecond=true                                                                 fd02::20   10.0.0.251   ready   
	                                                            k8s:id=app2                                                                                                        
	                                                            k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default                                             
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:zgroup=testapp                                                                                                 
	 374        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s1                                                                                 ready   
	                                                            k8s:node-role.kubernetes.io/control-plane                                                                          
	                                                            k8s:node.kubernetes.io/exclude-from-external-load-balancers                                                        
	                                                            reserved:host                                                                                                      
	 542        Disabled           Disabled          6110       k8s:app=prometheus                                                                 fd02::73   10.0.0.75    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=cilium-monitoring                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=prometheus-k8s                                                             
	                                                            k8s:io.kubernetes.pod.namespace=cilium-monitoring                                                                  
	 544        Disabled           Disabled          2443       k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default             fd02::1f   10.0.0.70    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                    
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:zgroup=testDSClient                                                                                            
	 1690       Disabled           Disabled          9378       k8s:id=app1                                                                        fd02::78   10.0.0.141   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default                                             
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:zgroup=testapp                                                                                                 
	 1731       Disabled           Disabled          7524       k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default             fd02::41   10.0.0.49    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                    
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:name=echo                                                                                                      
	 1925       Disabled           Disabled          34920      k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default             fd02::60   10.0.0.42    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                    
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:zgroup=testDS                                                                                                  
	 2361       Disabled           Disabled          4          reserved:health                                                                    fd02::74   10.0.0.221   ready   
	 3100       Disabled           Disabled          56434      k8s:id=app3                                                                        fd02::ab   10.0.0.60    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default                                             
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                           
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                    
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                            
	                                                            k8s:zgroup=testapp                                                                                                 
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-mfxmn -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID    Frontend                Service Type   Backend                            
	 1     10.96.0.10:53           ClusterIP      1 => 10.0.1.242:53 (active)        
	 2     10.96.0.10:9153         ClusterIP      1 => 10.0.1.242:9153 (active)      
	 3     10.105.101.62:3000      ClusterIP      1 => 10.0.0.225:3000 (active)      
	 4     10.107.193.248:9090     ClusterIP      1 => 10.0.0.75:9090 (active)       
	 6     10.96.0.1:443           ClusterIP      1 => 192.168.56.11:6443 (active)   
	 8     10.96.250.178:69        ClusterIP      1 => 10.0.0.141:69 (active)        
	                                              2 => 10.0.0.216:69 (active)        
	 9     10.96.250.178:80        ClusterIP      1 => 10.0.0.141:80 (active)        
	                                              2 => 10.0.0.216:80 (active)        
	 10    10.103.117.146:80       ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 11    10.103.117.146:69       ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 12    10.107.51.240:10080     ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 13    10.107.51.240:10069     ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 14    0.0.0.0:30091           NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 17    192.168.56.12:30091     NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 18    192.168.56.12:32718     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 21    0.0.0.0:32718           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 22    10.106.94.23:10080      ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 23    10.106.94.23:10069      ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 25    192.168.56.12:30011     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 26    0.0.0.0:30011           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 30    192.168.56.12:32706     NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 31    0.0.0.0:32706           NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 32    10.102.222.21:10069     ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 33    10.102.222.21:10080     ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 38    192.168.56.12:31538     NodePort       1 => 10.0.1.100:69 (active)        
	 39    192.168.56.12:31538/i   NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 40    0.0.0.0:31538           NodePort       1 => 10.0.1.100:69 (active)        
	 41    0.0.0.0:31538/i         NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 46    192.168.56.12:30494     NodePort       1 => 10.0.1.100:80 (active)        
	 47    192.168.56.12:30494/i   NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 48    0.0.0.0:30494           NodePort       1 => 10.0.1.100:80 (active)        
	 49    0.0.0.0:30494/i         NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 50    10.98.99.140:10080      ClusterIP      1 => 10.0.1.54:80 (active)         
	 51    10.98.99.140:10069      ClusterIP      1 => 10.0.1.54:69 (active)         
	 54    192.168.56.12:32642     NodePort       1 => 10.0.1.54:80 (active)         
	 55    192.168.56.12:32642/i   NodePort       1 => 10.0.1.54:80 (active)         
	 56    0.0.0.0:32642           NodePort       1 => 10.0.1.54:80 (active)         
	 57    0.0.0.0:32642/i         NodePort       1 => 10.0.1.54:80 (active)         
	 62    192.168.56.12:30975     NodePort       1 => 10.0.1.54:69 (active)         
	 63    192.168.56.12:30975/i   NodePort       1 => 10.0.1.54:69 (active)         
	 64    0.0.0.0:30975           NodePort       1 => 10.0.1.54:69 (active)         
	 65    0.0.0.0:30975/i         NodePort       1 => 10.0.1.54:69 (active)         
	 68    10.105.87.113:10069     ClusterIP      1 => 10.0.1.54:69 (active)         
	 69    10.105.87.113:10080     ClusterIP      1 => 10.0.1.54:80 (active)         
	 70    192.168.56.12:32104     NodePort       1 => 10.0.1.54:80 (active)         
	 71    0.0.0.0:32104           NodePort       1 => 10.0.1.54:80 (active)         
	 76    192.168.56.12:30713     NodePort       1 => 10.0.1.54:69 (active)         
	 77    0.0.0.0:30713           NodePort       1 => 10.0.1.54:69 (active)         
	 78    10.102.106.193:80       ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 79    0.0.0.0:31902           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 81    192.168.56.12:31902     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 83    10.99.164.251:80        ClusterIP      1 => 10.0.1.54:80 (active)         
	 84    0.0.0.0:30529           NodePort       1 => 10.0.1.54:80 (active)         
	 85    0.0.0.0:30529/i         NodePort       1 => 10.0.1.54:80 (active)         
	 90    192.168.56.12:30529     NodePort       1 => 10.0.1.54:80 (active)         
	 91    192.168.56.12:30529/i   NodePort       1 => 10.0.1.54:80 (active)         
	 92    10.109.187.42:20080     ClusterIP      1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 93    10.109.187.42:20069     ClusterIP      1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 94    192.0.2.233:20069       ExternalIPs    1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 95    192.0.2.233:20080       ExternalIPs    1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 96    192.168.56.12:31748     NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 97    0.0.0.0:31748           NodePort       1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 102   192.168.56.12:32629     NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 103   0.0.0.0:32629           NodePort       1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 104   10.106.131.128:69       ClusterIP      1 => 10.0.1.96:69 (active)         
	                                              2 => 10.0.0.49:69 (active)         
	 105   10.106.131.128:80       ClusterIP      1 => 10.0.1.96:80 (active)         
	                                              2 => 10.0.0.49:80 (active)         
	 107   192.168.56.12:30951     NodePort       1 => 10.0.1.96:80 (active)         
	                                              2 => 10.0.0.49:80 (active)         
	 109   0.0.0.0:30951           NodePort       1 => 10.0.1.96:80 (active)         
	                                              2 => 10.0.0.49:80 (active)         
	 112   192.168.56.12:30791     NodePort       1 => 10.0.1.96:69 (active)         
	                                              2 => 10.0.0.49:69 (active)         
	 113   0.0.0.0:30791           NodePort       1 => 10.0.1.96:69 (active)         
	                                              2 => 10.0.0.49:69 (active)         
	 114   [fd03::f7d5]:80         ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 115   [fd03::f7d5]:69         ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 116   [fd03::16fe]:10080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 117   [fd03::16fe]:10069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 118   [::]:32108              NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 120   [fd04::12]:32108        NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 122   [fd04::12]:30528        NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 123   [::]:30528              NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 124   [fd03::6a2a]:10080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 125   [fd03::6a2a]:10069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 127   [fd04::12]:30386        NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 128   [::]:30386              NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 130   [fd04::12]:30583        NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 131   [::]:30583              NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 132   [fd03::b838]:10069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 133   [fd03::b838]:10080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 136   [fd04::12]:31368        NodePort       1 => [fd02::198]:80 (active)       
	 137   [fd04::12]:31368/i      NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 138   [::]:31368              NodePort       1 => [fd02::198]:80 (active)       
	 139   [::]:31368/i            NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 142   [fd04::12]:32717        NodePort       1 => [fd02::198]:69 (active)       
	 143   [fd04::12]:32717/i      NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 144   [::]:32717              NodePort       1 => [fd02::198]:69 (active)       
	 145   [::]:32717/i            NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 146   [fd03::9e59]:10080      ClusterIP      1 => [fd02::1bb]:80 (active)       
	 147   [fd03::9e59]:10069      ClusterIP      1 => [fd02::1bb]:69 (active)       
	 150   [fd04::12]:30316        NodePort       1 => [fd02::1bb]:69 (active)       
	 151   [fd04::12]:30316/i      NodePort       1 => [fd02::1bb]:69 (active)       
	 152   [::]:30316              NodePort       1 => [fd02::1bb]:69 (active)       
	 153   [::]:30316/i            NodePort       1 => [fd02::1bb]:69 (active)       
	 154   [fd04::12]:30100        NodePort       1 => [fd02::1bb]:80 (active)       
	 155   [fd04::12]:30100/i      NodePort       1 => [fd02::1bb]:80 (active)       
	 158   [::]:30100              NodePort       1 => [fd02::1bb]:80 (active)       
	 159   [::]:30100/i            NodePort       1 => [fd02::1bb]:80 (active)       
	 160   [fd03::8f4]:10080       ClusterIP      1 => [fd02::1bb]:80 (active)       
	 161   [fd03::8f4]:10069       ClusterIP      1 => [fd02::1bb]:69 (active)       
	 163   [fd04::12]:31035        NodePort       1 => [fd02::1bb]:80 (active)       
	 164   [::]:31035              NodePort       1 => [fd02::1bb]:80 (active)       
	 166   [fd04::12]:31122        NodePort       1 => [fd02::1bb]:69 (active)       
	 167   [::]:31122              NodePort       1 => [fd02::1bb]:69 (active)       
	 168   [fd03::d8f4]:20080      ClusterIP      1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 169   [fd03::d8f4]:20069      ClusterIP      1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 170   [fd03::999]:20080       ExternalIPs    1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 171   [fd03::999]:20069       ExternalIPs    1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 173   [::]:32381              NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 174   [fd04::12]:32381        NodePort       1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 176   [fd04::12]:32356        NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 177   [::]:32356              NodePort       1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 180   192.168.56.12:8080      HostPort       1 => 10.0.1.54:80 (active)         
	 181   0.0.0.0:8080            HostPort       1 => 10.0.1.54:80 (active)         
	 183   [fd04::12]:8080         HostPort       1 => [fd02::1bb]:80 (active)       
	 184   [::]:8080               HostPort       1 => [fd02::1bb]:80 (active)       
	 185   192.168.56.12:6969      HostPort       1 => 10.0.1.54:69 (active)         
	 188   0.0.0.0:6969            HostPort       1 => 10.0.1.54:69 (active)         
	 190   [fd04::12]:6969         HostPort       1 => [fd02::1bb]:69 (active)       
	 191   [::]:6969               HostPort       1 => [fd02::1bb]:69 (active)       
	 192   192.168.56.11:20080     ExternalIPs    1 => 10.0.1.100:80 (active)        
	                                              2 => 10.0.0.42:80 (active)         
	 193   192.168.56.11:20069     ExternalIPs    1 => 10.0.1.100:69 (active)        
	                                              2 => 10.0.0.42:69 (active)         
	 194   [fd04::11]:20069        ExternalIPs    1 => [fd02::198]:69 (active)       
	                                              2 => [fd02::60]:69 (active)        
	 195   [fd04::11]:20080        ExternalIPs    1 => [fd02::198]:80 (active)       
	                                              2 => [fd02::60]:80 (active)        
	 196   10.106.137.186:443      ClusterIP      1 => 192.168.56.12:4244 (active)   
	                                              2 => 192.168.56.11:4244 (active)   
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-mfxmn -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                                                  IPv6        IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                                        
	 99         Disabled           Disabled          2443       k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default       fd02::1c1   10.0.1.36    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                       
	                                                            k8s:zgroup=testDSClient                                                                                       
	 148        Disabled           Disabled          10020      k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default       fd02::1bb   10.0.1.54    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                       
	                                                            k8s:zgroup=test-k8s2                                                                                          
	 161        Disabled           Disabled          4          reserved:health                                                              fd02::110   10.0.1.200   ready   
	 793        Disabled           Disabled          9803       k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=kube-system   fd02::140   10.0.1.242   ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns                                                               
	                                                            k8s:io.kubernetes.pod.namespace=kube-system                                                                   
	                                                            k8s:k8s-app=kube-dns                                                                                          
	 1078       Disabled           Disabled          7524       k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default       fd02::16e   10.0.1.96    ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                       
	                                                            k8s:name=echo                                                                                                 
	 2731       Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s2                                                                            ready   
	                                                            reserved:host                                                                                                 
	 3597       Disabled           Disabled          33690      k8s:io.cilium.k8s.namespace.labels.kubernetes.io/metadata.name=default       fd02::198   10.0.1.100   ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                               
	                                                            k8s:io.kubernetes.pod.namespace=default                                                                       
	                                                            k8s:zgroup=testDS                                                                                             
	 
Stderr:
 	 

===================== Exiting AfterFailed =====================
12:30:58 STEP: Running AfterEach for block EntireTestsuite

[[ATTACHMENT|e8d6a942_K8sDatapathServicesTest_Checks_N-S_loadbalancing_Tests_with_XDP,_vxlan_tunnel,_SNAT_and_Random.zip]]

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-net-next//100/artifact/687168f2_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-net-next//100/artifact/e8d6a942_K8sDatapathServicesTest_Checks_N-S_loadbalancing_Tests_with_XDP,_vxlan_tunnel,_SNAT_and_Random.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-net-next//100/artifact/test_results_Cilium-PR-K8s-1.26-kernel-net-next_100_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-net-next/100/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.
@maintainer-s-little-helper maintainer-s-little-helper bot added the ci/flake This is a known failure that occurs in the tree. Please investigate me! label Dec 6, 2022
@maintainer-s-little-helper maintainer-s-little-helper bot mentioned this issue Dec 6, 2022
Merged
@pchaigno
Copy link
Member

cc @brb

@pchaigno pchaigno added area/CI Continuous Integration testing issue or flake sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. labels Dec 12, 2022
@christarazi
Copy link
Member

Also occurred for

Test Name

K8sDatapathServicesTest Checks N/S loadbalancing Tests with XDP, direct routing, Hybrid and Random

Failure Output

FAIL: Can not connect to service "tftp://[fd04::11]:31438/hello" from outside cluster (3/10)

in #22846

@christarazi christarazi mentioned this issue Jan 4, 2023
Merged
@YutaroHayakawa YutaroHayakawa mentioned this issue Jan 19, 2023
Merged
@sayboras sayboras mentioned this issue Jan 25, 2023
Merged
@github-actions
Copy link

github-actions bot commented Mar 6, 2023

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Mar 6, 2023
@pchaigno pchaigno removed the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Mar 6, 2023
@giorio94 giorio94 mentioned this issue Mar 29, 2023
Merged
@github-actions
Copy link

github-actions bot commented May 6, 2023

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label May 6, 2023
@ysksuzuki ysksuzuki mentioned this issue May 6, 2023
Merged
8 tasks
@pchaigno pchaigno removed the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label May 6, 2023
@giorio94

This comment was marked as resolved.

Sign in to view
@giorio94 giorio94 mentioned this issue Jun 1, 2023
Merged
8 tasks
@giorio94
Copy link
Member

giorio94 commented Jun 1, 2023

The failure mentioned above was caused by https://cilium.slack.com/archives/C7PE7V806/p1685553432805279

@tklauser tklauser mentioned this issue Jun 7, 2023
Merged
1 task
@github-actions
Copy link

github-actions bot commented Aug 1, 2023

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Aug 1, 2023
@github-actions
Copy link

This issue has not seen any activity since it was marked stale.
Closing.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me! sig/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pchaigno @christarazi @giorio94