Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI: K8sPolicyTest Clusterwide policies Test clusterwide connectivity with policies #20793

Closed
maintainer-s-little-helper bot opened this issue Aug 5, 2022 · 6 comments
Closed

CI: K8sPolicyTest Clusterwide policies Test clusterwide connectivity with policies #20793

maintainer-s-little-helper bot opened this issue Aug 5, 2022 · 6 comments
Labels
ci/flake This is a known failure that occurs in the tree. Please investigate me! stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.

Comments

@maintainer-s-little-helper
Copy link

Test Name

K8sPolicyTest Clusterwide policies Test clusterwide connectivity with policies

Failure Output

FAIL: Ingress connectivity should be allowed for service in 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon namespace

Stacktrace

Click to show. 
/home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.9/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:527
Ingress connectivity should be allowed for service in 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon namespace
Expected command: kubectl exec -n 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon app2-5cc5d58844-7xr9l -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://10.105.47.62/public -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000017()', Connect: '0.000000',Transfer '0.000000', total '5.001676'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.9/src/github.com/cilium/cilium/test/k8s/net_policies.go:2413

Standard Output

Click to show. 
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 1
Number of "level=error" in logs: 0
⚠️  Number of "level=warning" in logs: 7
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
Top 3 errors/warnings:
Key allocation attempt failed
Cancelling obsolete endpoint creating due to new create for same pod
Creation of endpoint failed
Cilium pods: [cilium-22s22 cilium-9xndn]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                       Ingress   Egress
app3-6c7856c5b5-ptn9z     false     false
coredns-8cfc78c54-4hq52   false     false
app1-7b6ddb776f-zjhbw     false     false
app3-6c7856c5b5-dzwnm     false     false
app1-7b6ddb776f-rpjlw     false     false
app1-7b6ddb776f-w5pn9     false     false
app2-5cc5d58844-fm2jc     false     false
Cilium agent 'cilium-22s22': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 28 Failed 0
Cilium agent 'cilium-9xndn': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 55 Failed 0

Standard Error

Click to show. 
16:23:11 STEP: Running BeforeAll block for EntireTestsuite K8sPolicyTest Clusterwide policies
16:23:11 STEP: Deleting namespace 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon
16:23:11 STEP: Creating namespace 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon
16:23:11 STEP: Deleting namespace 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon
16:23:11 STEP: Creating namespace 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon
16:23:12 STEP: WaitforPods(namespace="2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp")
16:24:50 STEP: WaitforPods(namespace="2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp") => <nil>
16:24:50 STEP: WaitforPods(namespace="2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp")
16:26:08 STEP: WaitforPods(namespace="2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp") => <nil>
16:26:08 STEP: Applying Egress deny all clusterwide policy
16:26:44 STEP: Deleting Egress deny all clusterwide policy
16:26:49 STEP: Applying Ingress deny all clusterwide policy
16:26:55 STEP: Testing ingress connectivity from "2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
16:27:00 STEP: Testing ingress connectivity from "2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
16:27:05 STEP: Testing cross namespace connectivity from "2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon" to "2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
16:27:10 STEP: Testing cross namespace connectivity from "2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon" to "2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
16:27:15 STEP: Applying Egress deny all clusterwide policy
16:27:21 STEP: Applying Allow all clusterwide policy over ingress deny all and egress deny all
16:27:22 STEP: Testing ingress connectivity from "2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
FAIL: Ingress connectivity should be allowed for service in 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon namespace
Expected command: kubectl exec -n 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon app2-5cc5d58844-7xr9l -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://10.105.47.62/public -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000017()', Connect: '0.000000',Transfer '0.000000', total '5.001676'
Stderr:
 	 command terminated with exit code 28
	 

=== Test Finished at 2022-08-04T16:27:27Z====
16:27:27 STEP: Running JustAfterEach block for EntireTestsuite K8sPolicyTest
===================== TEST FAILED =====================
16:27:27 STEP: Running AfterFailed block for EntireTestsuite K8sPolicyTest
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0 
Stdout:
 	 NAMESPACE                                                         NAME                               READY   STATUS    RESTARTS   AGE     IP              NODE   NOMINATED NODE   READINESS GATES
	 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-rzmfd              2/2     Running   0          4m17s   10.0.0.128      k8s1   <none>           <none>
	 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-zjhbw              2/2     Running   0          4m17s   10.0.0.144      k8s1   <none>           <none>
	 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon   app2-5cc5d58844-7xr9l              1/1     Running   0          4m17s   10.0.0.80       k8s1   <none>           <none>
	 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon   app3-6c7856c5b5-dzwnm              1/1     Running   0          4m17s   10.0.0.50       k8s1   <none>           <none>
	 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-rpjlw              2/2     Running   0          2m39s   10.0.0.75       k8s1   <none>           <none>
	 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-w5pn9              2/2     Running   0          2m39s   10.0.0.130      k8s1   <none>           <none>
	 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon   app2-5cc5d58844-fm2jc              1/1     Running   0          2m39s   10.0.0.149      k8s1   <none>           <none>
	 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon   app3-6c7856c5b5-ptn9z              1/1     Running   0          2m39s   10.0.0.28       k8s1   <none>           <none>
	 cilium-monitoring                                                 grafana-7fd557d749-jkcvf           0/1     Running   0          84m     10.0.0.126      k8s2   <none>           <none>
	 cilium-monitoring                                                 prometheus-d87f8f984-p4vwb         1/1     Running   0          84m     10.0.0.30       k8s2   <none>           <none>
	 kube-system                                                       cilium-22s22                       1/1     Running   0          8m6s    192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       cilium-9xndn                       1/1     Running   0          8m6s    192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       cilium-operator-8648574975-bxvst   1/1     Running   0          8m6s    192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       cilium-operator-8648574975-cl2nf   1/1     Running   0          8m6s    192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       coredns-8cfc78c54-4hq52            1/1     Running   0          30m     10.0.1.121      k8s2   <none>           <none>
	 kube-system                                                       etcd-k8s1                          1/1     Running   0          87m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-apiserver-k8s1                1/1     Running   0          87m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-controller-manager-k8s1       1/1     Running   4          88m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-proxy-487hf                   1/1     Running   0          86m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-proxy-gjv2q                   1/1     Running   0          85m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       kube-scheduler-k8s1                1/1     Running   5          88m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       log-gatherer-jgpjn                 1/1     Running   0          84m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       log-gatherer-z8f8x                 1/1     Running   0          84m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       registry-adder-k4k59               1/1     Running   0          85m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       registry-adder-sfhf7               1/1     Running   0          85m     192.168.56.11   k8s1   <none>           <none>
	 
Stderr:
 	 

Fetching command output from pods [cilium-22s22 cilium-9xndn]
cmd: kubectl exec -n kube-system cilium-22s22 -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend             Service Type   Backend                            
	 1    10.96.0.10:53        ClusterIP      1 => 10.0.1.121:53 (active)        
	 2    10.96.0.10:9153      ClusterIP      1 => 10.0.1.121:9153 (active)      
	 3    10.106.68.202:9090   ClusterIP      1 => 10.0.0.30:9090 (active)       
	 5    10.103.72.139:3000   ClusterIP                                         
	 6    10.96.0.1:443        ClusterIP      1 => 192.168.56.11:6443 (active)   
	 27   10.104.41.67:443     ClusterIP      1 => 192.168.56.12:4244 (active)   
	                                          2 => 192.168.56.11:4244 (active)   
	 37   10.105.47.62:80      ClusterIP      1 => 10.0.0.128:80 (active)        
	                                          2 => 10.0.0.144:80 (active)        
	 38   10.105.47.62:69      ClusterIP      1 => 10.0.0.128:69 (active)        
	                                          2 => 10.0.0.144:69 (active)        
	 39   10.106.205.167:80    ClusterIP      1 => 10.0.0.75:80 (active)         
	                                          2 => 10.0.0.130:80 (active)        
	 40   10.106.205.167:69    ClusterIP      1 => 10.0.0.75:69 (active)         
	                                          2 => 10.0.0.130:69 (active)        
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-22s22 -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                       IPv6        IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                             
	 243        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s2                                                 ready   
	                                                            reserved:host                                                                      
	 1133       Enabled            Enabled           2112       k8s:io.cilium.k8s.policy.cluster=default          fd02::1ed   10.0.1.121   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns                                    
	                                                            k8s:io.kubernetes.pod.namespace=kube-system                                        
	                                                            k8s:k8s-app=kube-dns                                                               
	 2014       Enabled            Enabled           4          reserved:health                                   fd02::12b   10.0.1.27    ready   
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-9xndn -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend             Service Type   Backend                            
	 2    10.96.0.10:53        ClusterIP      1 => 10.0.1.121:53 (active)        
	 3    10.96.0.10:9153      ClusterIP      1 => 10.0.1.121:9153 (active)      
	 4    10.103.72.139:3000   ClusterIP                                         
	 5    10.96.0.1:443        ClusterIP      1 => 192.168.56.11:6443 (active)   
	 6    10.106.68.202:9090   ClusterIP      1 => 10.0.0.30:9090 (active)       
	 27   10.104.41.67:443     ClusterIP      1 => 192.168.56.12:4244 (active)   
	                                          2 => 192.168.56.11:4244 (active)   
	 37   10.105.47.62:80      ClusterIP      1 => 10.0.0.128:80 (active)        
	                                          2 => 10.0.0.144:80 (active)        
	 38   10.105.47.62:69      ClusterIP      1 => 10.0.0.128:69 (active)        
	                                          2 => 10.0.0.144:69 (active)        
	 39   10.106.205.167:69    ClusterIP      1 => 10.0.0.75:69 (active)         
	                                          2 => 10.0.0.130:69 (active)        
	 40   10.106.205.167:80    ClusterIP      1 => 10.0.0.75:80 (active)         
	                                          2 => 10.0.0.130:80 (active)        
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-9xndn -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                                                                                  IPv6       IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                                                                       
	 11         Enabled            Enabled           1715       k8s:appSecond=true                                                                                           fd02::c1   10.0.0.80    ready   
	                                                            k8s:id=app2                                                                                                                                  
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 118        Enabled            Enabled           51117      k8s:id=app3                                                                                                  fd02::12   10.0.0.28    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                                              
	                                                            k8s:io.kubernetes.pod.namespace=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 119        Enabled            Enabled           43316      k8s:appSecond=true                                                                                           fd02::80   10.0.0.149   ready   
	                                                            k8s:id=app2                                                                                                                                  
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 189        Enabled            Enabled           4          reserved:health                                                                                              fd02::7    10.0.0.197   ready   
	 685        Enabled            Enabled           12853      k8s:id=app1                                                                                                  fd02::92   10.0.0.128   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 743        Enabled            Enabled           12853      k8s:id=app1                                                                                                  fd02::7d   10.0.0.144   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 1127       Enabled            Enabled           19909      k8s:id=app1                                                                                                  fd02::8    10.0.0.75    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 1804       Enabled            Enabled           33563      k8s:id=app3                                                                                                  fd02::e7   10.0.0.50    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                                              
	                                                            k8s:io.kubernetes.pod.namespace=2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 2368       Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s1                                                                                                           ready   
	                                                            k8s:node-role.kubernetes.io/master                                                                                                           
	                                                            reserved:host                                                                                                                                
	 3891       Enabled            Enabled           19909      k8s:id=app1                                                                                                  fd02::f8   10.0.0.130   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 
Stderr:
 	 

===================== Exiting AfterFailed =====================
16:27:37 STEP: Running AfterEach for block EntireTestsuite

[[ATTACHMENT|2b280443_K8sPolicyTest_Clusterwide_policies_Test_clusterwide_connectivity_with_policies.zip]]
16:27:39 STEP: Running AfterAll block for EntireTestsuite K8sPolicyTest Clusterwide policies
16:27:39 STEP: Deleting namespace 2022080416231k8spolicytestclusterwidepoliciestestclusterwidecon
16:27:51 STEP: Deleting namespace 2022080416232k8spolicytestclusterwidepoliciestestclusterwidecon

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2261/artifact/2b280443_K8sPolicyTest_Clusterwide_policies_Test_clusterwide_connectivity_with_policies.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2261/artifact/test_results_Cilium-PR-K8s-1.16-kernel-4.9_2261_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9/2261/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.
@maintainer-s-little-helper maintainer-s-little-helper bot added the ci/flake This is a known failure that occurs in the tree. Please investigate me! label Aug 5, 2022
@maintainer-s-little-helper maintainer-s-little-helper bot mentioned this issue Aug 5, 2022
Merged
@maintainer-s-little-helper
Copy link
Author

PR #20861 hit this flake with 87.27% similarity:

Click to show.

Test Name

K8sPolicyTest Clusterwide policies Test clusterwide connectivity with policies

Failure Output

FAIL: Ingress connectivity should be allowed for service in 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon namespace

Stacktrace

Click to show. 
/home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.9/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:527
Ingress connectivity should be allowed for service in 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon namespace
Expected command: kubectl exec -n 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon app2-5cc5d58844-jt5xh -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://10.111.155.120/public -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000336()', Connect: '0.000000',Transfer '0.000000', total '5.000789'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.9/src/github.com/cilium/cilium/test/k8s/net_policies.go:2419

Standard Output

Click to show. 
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 2
Number of "level=error" in logs: 0
⚠️  Number of "level=warning" in logs: 7
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
Top 3 errors/warnings:
Cancelling obsolete endpoint creating due to new create for same pod
Creation of endpoint failed
Key allocation attempt failed
Cilium pods: [cilium-hxdgl cilium-rj2sv]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                       Ingress   Egress
coredns-8cfc78c54-m684q   false     false
app1-7b6ddb776f-vh6hs     false     false
app1-7b6ddb776f-w2jvs     false     false
app2-5cc5d58844-skzrf     false     false
app3-6c7856c5b5-mhnjr     false     false
app1-7b6ddb776f-8qffg     false     false
Cilium agent 'cilium-hxdgl': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 29 Failed 0
Cilium agent 'cilium-rj2sv': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 56 Failed 0

Standard Error

Click to show. 
19:27:28 STEP: Running BeforeAll block for EntireTestsuite K8sPolicyTest Clusterwide policies
19:27:28 STEP: Deleting namespace 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon
19:27:28 STEP: Creating namespace 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon
19:27:28 STEP: Deleting namespace 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon
19:27:28 STEP: Creating namespace 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon
19:27:29 STEP: WaitforPods(namespace="2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp")
19:27:38 STEP: WaitforPods(namespace="2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp") => <nil>
19:27:38 STEP: WaitforPods(namespace="2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp")
19:29:17 STEP: WaitforPods(namespace="2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp") => <nil>
19:29:17 STEP: Applying Egress deny all clusterwide policy
19:29:54 STEP: Deleting Egress deny all clusterwide policy
19:30:00 STEP: Applying Ingress deny all clusterwide policy
19:30:07 STEP: Testing ingress connectivity from "2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
19:30:12 STEP: Testing ingress connectivity from "2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
19:30:17 STEP: Testing cross namespace connectivity from "2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon" to "2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
19:30:22 STEP: Testing cross namespace connectivity from "2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon" to "2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
19:30:27 STEP: Applying Egress deny all clusterwide policy
19:30:34 STEP: Applying Allow all clusterwide policy over ingress deny all and egress deny all
19:30:34 STEP: Testing ingress connectivity from "2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
19:30:34 STEP: Testing ingress connectivity from "2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
FAIL: Ingress connectivity should be allowed for service in 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon namespace
Expected command: kubectl exec -n 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon app2-5cc5d58844-jt5xh -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://10.111.155.120/public -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000336()', Connect: '0.000000',Transfer '0.000000', total '5.000789'
Stderr:
 	 command terminated with exit code 28
	 

=== Test Finished at 2022-08-10T19:30:40Z====
19:30:40 STEP: Running JustAfterEach block for EntireTestsuite K8sPolicyTest
===================== TEST FAILED =====================
19:30:40 STEP: Running AfterFailed block for EntireTestsuite K8sPolicyTest
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0 
Stdout:
 	 NAMESPACE                                                         NAME                              READY   STATUS    RESTARTS   AGE     IP              NODE   NOMINATED NODE   READINESS GATES
	 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-vh6hs             2/2     Running   0          3m13s   10.0.0.6        k8s1   <none>           <none>
	 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-w2jvs             2/2     Running   0          3m13s   10.0.0.165      k8s1   <none>           <none>
	 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon   app2-5cc5d58844-skzrf             1/1     Running   0          3m13s   10.0.0.31       k8s1   <none>           <none>
	 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon   app3-6c7856c5b5-mhnjr             1/1     Running   0          3m13s   10.0.0.11       k8s1   <none>           <none>
	 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-8qffg             2/2     Running   0          3m4s    10.0.0.234      k8s1   <none>           <none>
	 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-h8hl2             2/2     Running   0          3m4s    10.0.0.192      k8s1   <none>           <none>
	 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon   app2-5cc5d58844-jt5xh             1/1     Running   0          3m4s    10.0.0.235      k8s1   <none>           <none>
	 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon   app3-6c7856c5b5-jdrhk             1/1     Running   0          3m4s    10.0.0.254      k8s1   <none>           <none>
	 cilium-monitoring                                                 grafana-7fd557d749-n4vpx          0/1     Running   0          78m     10.0.0.180      k8s2   <none>           <none>
	 cilium-monitoring                                                 prometheus-d87f8f984-5wcqk        1/1     Running   0          78m     10.0.0.80       k8s2   <none>           <none>
	 kube-system                                                       cilium-hxdgl                      1/1     Running   0          6m53s   192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       cilium-operator-b466cfd59-mxj9m   1/1     Running   0          6m53s   192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       cilium-operator-b466cfd59-z7gmc   1/1     Running   0          6m53s   192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       cilium-rj2sv                      1/1     Running   0          6m53s   192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       coredns-8cfc78c54-m684q           1/1     Running   0          37m     10.0.1.227      k8s2   <none>           <none>
	 kube-system                                                       etcd-k8s1                         1/1     Running   0          80m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-apiserver-k8s1               1/1     Running   0          80m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-controller-manager-k8s1      1/1     Running   2          80m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-proxy-4l69h                  1/1     Running   0          81m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-proxy-tr6t8                  1/1     Running   0          79m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       kube-scheduler-k8s1               1/1     Running   2          80m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       log-gatherer-g5wpt                1/1     Running   0          78m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       log-gatherer-m89z2                1/1     Running   0          78m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       registry-adder-ff7j7              1/1     Running   0          79m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       registry-adder-mc9k7              1/1     Running   0          79m     192.168.56.11   k8s1   <none>           <none>
	 
Stderr:
 	 

Fetching command output from pods [cilium-hxdgl cilium-rj2sv]
cmd: kubectl exec -n kube-system cilium-hxdgl -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend              Service Type   Backend                            
	 1    10.96.0.1:443         ClusterIP      1 => 192.168.56.11:6443 (active)   
	 2    10.96.0.10:53         ClusterIP      1 => 10.0.1.227:53 (active)        
	 3    10.96.0.10:9153       ClusterIP      1 => 10.0.1.227:9153 (active)      
	 5    10.109.255.131:3000   ClusterIP                                         
	 6    10.110.119.245:9090   ClusterIP      1 => 10.0.0.80:9090 (active)       
	 27   10.103.96.147:443     ClusterIP      1 => 192.168.56.11:4244 (active)   
	                                           2 => 192.168.56.12:4244 (active)   
	 37   10.105.202.254:69     ClusterIP      1 => 10.0.0.6:69 (active)          
	                                           2 => 10.0.0.165:69 (active)        
	 38   10.105.202.254:80     ClusterIP      1 => 10.0.0.6:80 (active)          
	                                           2 => 10.0.0.165:80 (active)        
	 39   10.111.155.120:80     ClusterIP      1 => 10.0.0.234:80 (active)        
	                                           2 => 10.0.0.192:80 (active)        
	 40   10.111.155.120:69     ClusterIP      1 => 10.0.0.234:69 (active)        
	                                           2 => 10.0.0.192:69 (active)        
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-hxdgl -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                       IPv6        IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                             
	 906        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s2                                                 ready   
	                                                            reserved:host                                                                      
	 1210       Enabled            Enabled           4          reserved:health                                   fd02::1f8   10.0.1.168   ready   
	 1395       Enabled            Enabled           349        k8s:io.cilium.k8s.policy.cluster=default          fd02::145   10.0.1.227   ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns                                    
	                                                            k8s:io.kubernetes.pod.namespace=kube-system                                        
	                                                            k8s:k8s-app=kube-dns                                                               
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-rj2sv -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend              Service Type   Backend                            
	 1    10.96.0.10:53         ClusterIP      1 => 10.0.1.227:53 (active)        
	 2    10.96.0.10:9153       ClusterIP      1 => 10.0.1.227:9153 (active)      
	 3    10.109.255.131:3000   ClusterIP                                         
	 4    10.110.119.245:9090   ClusterIP      1 => 10.0.0.80:9090 (active)       
	 6    10.96.0.1:443         ClusterIP      1 => 192.168.56.11:6443 (active)   
	 27   10.103.96.147:443     ClusterIP      1 => 192.168.56.11:4244 (active)   
	                                           2 => 192.168.56.12:4244 (active)   
	 37   10.105.202.254:80     ClusterIP      1 => 10.0.0.6:80 (active)          
	                                           2 => 10.0.0.165:80 (active)        
	 38   10.105.202.254:69     ClusterIP      1 => 10.0.0.6:69 (active)          
	                                           2 => 10.0.0.165:69 (active)        
	 39   10.111.155.120:80     ClusterIP      1 => 10.0.0.234:80 (active)        
	                                           2 => 10.0.0.192:80 (active)        
	 40   10.111.155.120:69     ClusterIP      1 => 10.0.0.234:69 (active)        
	                                           2 => 10.0.0.192:69 (active)        
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-rj2sv -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                                                                                  IPv6       IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                                                                       
	 312        Enabled            Enabled           21689      k8s:id=app1                                                                                                  fd02::e    10.0.0.165   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 443        Enabled            Enabled           11090      k8s:id=app3                                                                                                  fd02::6d   10.0.0.11    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                                              
	                                                            k8s:io.kubernetes.pod.namespace=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 612        Enabled            Enabled           4464       k8s:id=app1                                                                                                  fd02::ef   10.0.0.234   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 842        Enabled            Enabled           63135      k8s:appSecond=true                                                                                           fd02::94   10.0.0.31    ready   
	                                                            k8s:id=app2                                                                                                                                  
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 1034       Enabled            Enabled           9394       k8s:appSecond=true                                                                                           fd02::7    10.0.0.235   ready   
	                                                            k8s:id=app2                                                                                                                                  
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 1453       Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s1                                                                                                           ready   
	                                                            k8s:node-role.kubernetes.io/master                                                                                                           
	                                                            k8s:status=lockdown                                                                                                                          
	                                                            reserved:host                                                                                                                                
	 2122       Enabled            Enabled           21689      k8s:id=app1                                                                                                  fd02::ac   10.0.0.6     ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 2247       Enabled            Enabled           3490       k8s:id=app3                                                                                                  fd02::74   10.0.0.254   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                                              
	                                                            k8s:io.kubernetes.pod.namespace=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 2742       Enabled            Enabled           4          reserved:health                                                                                              fd02::41   10.0.0.81    ready   
	 2943       Enabled            Enabled           4464       k8s:id=app1                                                                                                  fd02::99   10.0.0.192   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                     
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                         
	                                                            k8s:io.kubernetes.pod.namespace=2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon                                              
	                                                            k8s:zgroup=testapp                                                                                                                           
	 
Stderr:
 	 

===================== Exiting AfterFailed =====================
19:30:50 STEP: Running AfterEach for block EntireTestsuite

[[ATTACHMENT|982eca99_K8sPolicyTest_Clusterwide_policies_Test_clusterwide_connectivity_with_policies.zip]]
19:30:51 STEP: Running AfterAll block for EntireTestsuite K8sPolicyTest Clusterwide policies
19:30:52 STEP: Deleting namespace 2022081019271k8spolicytestclusterwidepoliciestestclusterwidecon
19:31:03 STEP: Deleting namespace 2022081019272k8spolicytestclusterwidepoliciestestclusterwidecon

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/0923b07f_K8sDatapathConfig_Host_firewall_With_VXLAN_and_endpoint_routes.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/2bb34214_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_TFTP_with_DNS_Proxy_port_collision_Tests_TFTP_from_DNS_Proxy_Port.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/53bab7b3_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_with_L7_policy_Tests_NodePort_with_L7_Policy.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/5711e265_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Tests_NodePort_inside_cluster_(kube-proxy)_with_externalTrafficPolicy=Local.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/6f749ef4_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Checks_service_on_same_node.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/78c70be9_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Tests_NodePort_inside_cluster_(kube-proxy)_with_IPSec_and_externalTrafficPolicy=Local.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/79dc88b1_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Tests_NodePort_inside_cluster_(kube-proxy)_with_the_host_firewall_and_externalTrafficPolicy=Local.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/982eca99_K8sPolicyTest_Clusterwide_policies_Test_clusterwide_connectivity_with_policies.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/9ce12401_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_with_L4_policy_Tests_NodePort_with_L4_Policy.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/b2512fc1_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Checks_service_accessing_itself_(hairpin_flow).zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/bfa71621_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Checks_in-cluster_KPR.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/de97162e_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Tests_NodePort_inside_cluster_(kube-proxy)_.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/e7a3bb82_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Checks_connectivity_when_skipping_socket_lb_in_pod_ns.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/e8c5e25c_K8sServicesTest_Checks_E-W_loadbalancing_(ClusterIP,_NodePort_from_inside_cluster,_etc)_Checks_DualStack_services.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2308/artifact/test_results_Cilium-PR-K8s-1.16-kernel-4.9_2308_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9/2308/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

Merged
@maintainer-s-little-helper
Copy link
Author

PR #20524 hit this flake with 88.38% similarity:

Click to show.

Test Name

K8sPolicyTest Clusterwide policies Test clusterwide connectivity with policies

Failure Output

FAIL: Ingress connectivity should be allowed for service in 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon namespace

Stacktrace

Click to show. 
/home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.9/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:527
Ingress connectivity should be allowed for service in 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon namespace
Expected command: kubectl exec -n 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon app2-5cc5d58844-xc6cd -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://10.97.95.168/public -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000016()', Connect: '0.000000',Transfer '0.000000', total '5.001203'
Stderr:
 	 command terminated with exit code 28
	 

/home/jenkins/workspace/Cilium-PR-K8s-1.16-kernel-4.9/src/github.com/cilium/cilium/test/k8s/net_policies.go:2419

Standard Output

Click to show. 
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
Number of "level=warning" in logs: 0
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
No errors/warnings found in logs
Number of "context deadline exceeded" in logs: 0
Number of "level=error" in logs: 0
⚠️  Number of "level=warning" in logs: 6
Number of "Cilium API handler panicked" in logs: 0
Number of "Goroutine took lock for more than" in logs: 0
Top 3 errors/warnings:
Key allocation attempt failed
Cancelling obsolete endpoint creating due to new create for same pod
Creation of endpoint failed
Cilium pods: [cilium-b5jlm cilium-gzcf9]
Netpols loaded: 
CiliumNetworkPolicies loaded: 
Endpoint Policy Enforcement:
Pod                          Ingress   Egress
app1-7b6ddb776f-npvdm        false     false
app1-7b6ddb776f-vbldp        false     false
app3-6c7856c5b5-48ldb        false     false
app2-5cc5d58844-xc6cd        false     false
coredns-8cfc78c54-bc67x      false     false
app2-5cc5d58844-fs65t        false     false
app1-7b6ddb776f-crnbb        false     false
grafana-7fd557d749-5t97g     false     false
prometheus-d87f8f984-46q64   false     false
Cilium agent 'cilium-b5jlm': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 56 Failed 0
Cilium agent 'cilium-gzcf9': Status: Ok  Health: Ok Nodes "" ContainerRuntime:  Kubernetes: Ok KVstore: Ok Controllers: Total 39 Failed 0

Standard Error

Click to show. 
01:53:11 STEP: Running BeforeAll block for EntireTestsuite K8sPolicyTest Clusterwide policies
01:53:11 STEP: Deleting namespace 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon
01:53:12 STEP: Creating namespace 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon
01:53:12 STEP: Deleting namespace 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon
01:53:12 STEP: Creating namespace 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon
01:53:12 STEP: WaitforPods(namespace="2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp")
01:53:22 STEP: WaitforPods(namespace="2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp") => <nil>
01:53:23 STEP: WaitforPods(namespace="2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp")
01:55:03 STEP: WaitforPods(namespace="2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon", filter="-l zgroup=testapp") => <nil>
01:55:03 STEP: Applying Egress deny all clusterwide policy
01:55:40 STEP: Deleting Egress deny all clusterwide policy
01:55:47 STEP: Applying Ingress deny all clusterwide policy
01:55:54 STEP: Testing ingress connectivity from "2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
01:55:59 STEP: Testing ingress connectivity from "2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
01:56:04 STEP: Testing cross namespace connectivity from "2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon" to "2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
01:56:09 STEP: Testing cross namespace connectivity from "2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon" to "2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
01:56:14 STEP: Applying Egress deny all clusterwide policy
01:56:21 STEP: Applying Allow all clusterwide policy over ingress deny all and egress deny all
01:56:22 STEP: Testing ingress connectivity from "2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
01:56:22 STEP: Testing ingress connectivity from "2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon" namespace
FAIL: Ingress connectivity should be allowed for service in 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon namespace
Expected command: kubectl exec -n 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon app2-5cc5d58844-xc6cd -- curl --path-as-is -s -D /dev/stderr --fail --connect-timeout 5 --max-time 20 http://10.97.95.168/public -w "time-> DNS: '%{time_namelookup}(%{remote_ip})', Connect: '%{time_connect}',Transfer '%{time_starttransfer}', total '%{time_total}'" 
To succeed, but it failed:
Exitcode: 28 
Err: exit status 28
Stdout:
 	 time-> DNS: '0.000016()', Connect: '0.000000',Transfer '0.000000', total '5.001203'
Stderr:
 	 command terminated with exit code 28
	 

=== Test Finished at 2022-08-12T01:56:27Z====
01:56:27 STEP: Running JustAfterEach block for EntireTestsuite K8sPolicyTest
===================== TEST FAILED =====================
01:56:28 STEP: Running AfterFailed block for EntireTestsuite K8sPolicyTest
cmd: kubectl get pods -o wide --all-namespaces
Exitcode: 0 
Stdout:
 	 NAMESPACE                                                         NAME                               READY   STATUS    RESTARTS   AGE     IP              NODE   NOMINATED NODE   READINESS GATES
	 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-npvdm              2/2     Running   0          3m18s   10.0.1.145      k8s1   <none>           <none>
	 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-vbldp              2/2     Running   0          3m18s   10.0.1.250      k8s1   <none>           <none>
	 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon   app2-5cc5d58844-fs65t              1/1     Running   0          3m18s   10.0.1.29       k8s1   <none>           <none>
	 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon   app3-6c7856c5b5-48ldb              1/1     Running   0          3m18s   10.0.1.169      k8s1   <none>           <none>
	 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-crnbb              2/2     Running   0          3m8s    10.0.1.58       k8s1   <none>           <none>
	 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon   app1-7b6ddb776f-wmvbw              2/2     Running   0          3m8s    10.0.1.69       k8s1   <none>           <none>
	 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon   app2-5cc5d58844-xc6cd              1/1     Running   0          3m8s    10.0.1.165      k8s1   <none>           <none>
	 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon   app3-6c7856c5b5-k98f5              1/1     Running   0          3m7s    10.0.1.237      k8s1   <none>           <none>
	 cilium-monitoring                                                 grafana-7fd557d749-5t97g           1/1     Running   0          83m     10.0.0.156      k8s2   <none>           <none>
	 cilium-monitoring                                                 prometheus-d87f8f984-46q64         1/1     Running   0          83m     10.0.0.103      k8s2   <none>           <none>
	 kube-system                                                       cilium-b5jlm                       1/1     Running   0          8m44s   192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       cilium-gzcf9                       1/1     Running   0          8m44s   192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       cilium-operator-5979bdd6b5-blqkj   1/1     Running   0          8m44s   192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       cilium-operator-5979bdd6b5-wvvxm   1/1     Running   0          8m44s   192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       coredns-8cfc78c54-bc67x            1/1     Running   0          52m     10.0.0.17       k8s2   <none>           <none>
	 kube-system                                                       etcd-k8s1                          1/1     Running   0          86m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-apiserver-k8s1                1/1     Running   0          85m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-controller-manager-k8s1       1/1     Running   3          86m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-proxy-bz8ll                   1/1     Running   0          86m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       kube-proxy-cf8zm                   1/1     Running   0          83m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       kube-scheduler-k8s1                1/1     Running   3          85m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       log-gatherer-72blk                 1/1     Running   0          83m     192.168.56.12   k8s2   <none>           <none>
	 kube-system                                                       log-gatherer-xt6mm                 1/1     Running   0          83m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       registry-adder-bz7s4               1/1     Running   0          83m     192.168.56.11   k8s1   <none>           <none>
	 kube-system                                                       registry-adder-gpzq2               1/1     Running   0          83m     192.168.56.12   k8s2   <none>           <none>
	 
Stderr:
 	 

Fetching command output from pods [cilium-b5jlm cilium-gzcf9]
cmd: kubectl exec -n kube-system cilium-b5jlm -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend            Service Type   Backend                            
	 1    10.105.3.71:3000    ClusterIP      1 => 10.0.0.156:3000 (active)      
	 2    10.97.11.103:9090   ClusterIP      1 => 10.0.0.103:9090 (active)      
	 4    10.96.0.1:443       ClusterIP      1 => 192.168.56.11:6443 (active)   
	 5    10.96.0.10:53       ClusterIP      1 => 10.0.0.17:53 (active)         
	 6    10.96.0.10:9153     ClusterIP      1 => 10.0.0.17:9153 (active)       
	 27   10.102.23.130:443   ClusterIP      1 => 192.168.56.12:4244 (active)   
	                                         2 => 192.168.56.11:4244 (active)   
	 37   10.97.107.102:80    ClusterIP      1 => 10.0.1.145:80 (active)        
	                                         2 => 10.0.1.250:80 (active)        
	 38   10.97.107.102:69    ClusterIP      1 => 10.0.1.145:69 (active)        
	                                         2 => 10.0.1.250:69 (active)        
	 39   10.97.95.168:80     ClusterIP      1 => 10.0.1.58:80 (active)         
	                                         2 => 10.0.1.69:80 (active)         
	 40   10.97.95.168:69     ClusterIP      1 => 10.0.1.58:69 (active)         
	                                         2 => 10.0.1.69:69 (active)         
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-b5jlm -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                                                                                  IPv6        IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                                                                        
	 306        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s1                                                                                                            ready   
	                                                            k8s:node-role.kubernetes.io/master                                                                                                            
	                                                            reserved:host                                                                                                                                 
	 572        Enabled            Enabled           8957       k8s:appSecond=true                                                                                           fd02::172   10.0.1.29    ready   
	                                                            k8s:id=app2                                                                                                                                   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                                                          
	                                                            k8s:io.kubernetes.pod.namespace=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 724        Enabled            Enabled           30397      k8s:appSecond=true                                                                                           fd02::1de   10.0.1.165   ready   
	                                                            k8s:id=app2                                                                                                                                   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app2-account                                                                                          
	                                                            k8s:io.kubernetes.pod.namespace=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 790        Enabled            Enabled           4          reserved:health                                                                                              fd02::1bb   10.0.1.206   ready   
	 841        Enabled            Enabled           52061      k8s:id=app1                                                                                                  fd02::1d0   10.0.1.250   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                          
	                                                            k8s:io.kubernetes.pod.namespace=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 916        Enabled            Enabled           36707      k8s:id=app1                                                                                                  fd02::198   10.0.1.69    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                          
	                                                            k8s:io.kubernetes.pod.namespace=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 1033       Enabled            Enabled           52061      k8s:id=app1                                                                                                  fd02::186   10.0.1.145   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                          
	                                                            k8s:io.kubernetes.pod.namespace=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 1373       Enabled            Enabled           36707      k8s:id=app1                                                                                                  fd02::1e8   10.0.1.58    ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=app1-account                                                                                          
	                                                            k8s:io.kubernetes.pod.namespace=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 2302       Enabled            Enabled           60281      k8s:id=app3                                                                                                  fd02::171   10.0.1.237   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                                               
	                                                            k8s:io.kubernetes.pod.namespace=2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 2322       Enabled            Enabled           36988      k8s:id=app3                                                                                                  fd02::176   10.0.1.169   ready   
	                                                            k8s:io.cilium.k8s.namespace.labels.nslabel=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                    
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                                                                      
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                                                                               
	                                                            k8s:io.kubernetes.pod.namespace=2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon                                               
	                                                            k8s:zgroup=testapp                                                                                                                            
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-gzcf9 -c cilium-agent -- cilium service list
Exitcode: 0 
Stdout:
 	 ID   Frontend            Service Type   Backend                            
	 1    10.96.0.1:443       ClusterIP      1 => 192.168.56.11:6443 (active)   
	 2    10.96.0.10:53       ClusterIP      1 => 10.0.0.17:53 (active)         
	 3    10.96.0.10:9153     ClusterIP      1 => 10.0.0.17:9153 (active)       
	 6    10.105.3.71:3000    ClusterIP      1 => 10.0.0.156:3000 (active)      
	 7    10.97.11.103:9090   ClusterIP      1 => 10.0.0.103:9090 (active)      
	 28   10.102.23.130:443   ClusterIP      1 => 192.168.56.12:4244 (active)   
	                                         2 => 192.168.56.11:4244 (active)   
	 38   10.97.107.102:69    ClusterIP      1 => 10.0.1.145:69 (active)        
	                                         2 => 10.0.1.250:69 (active)        
	 39   10.97.107.102:80    ClusterIP      1 => 10.0.1.145:80 (active)        
	                                         2 => 10.0.1.250:80 (active)        
	 40   10.97.95.168:80     ClusterIP      1 => 10.0.1.58:80 (active)         
	                                         2 => 10.0.1.69:80 (active)         
	 41   10.97.95.168:69     ClusterIP      1 => 10.0.1.58:69 (active)         
	                                         2 => 10.0.1.69:69 (active)         
	 
Stderr:
 	 

cmd: kubectl exec -n kube-system cilium-gzcf9 -c cilium-agent -- cilium endpoint list
Exitcode: 0 
Stdout:
 	 ENDPOINT   POLICY (ingress)   POLICY (egress)   IDENTITY   LABELS (source:key[=value])                              IPv6       IPv4         STATUS   
	            ENFORCEMENT        ENFORCEMENT                                                                                                   
	 33         Enabled            Enabled           36837      k8s:app=grafana                                          fd02::9a   10.0.0.156   ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                 
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=default                                          
	                                                            k8s:io.kubernetes.pod.namespace=cilium-monitoring                                        
	 423        Disabled           Disabled          1          k8s:cilium.io/ci-node=k8s2                                                       ready   
	                                                            reserved:host                                                                            
	 456        Enabled            Enabled           4          reserved:health                                          fd02::23   10.0.0.19    ready   
	 1592       Enabled            Enabled           6524       k8s:io.cilium.k8s.policy.cluster=default                 fd02::14   10.0.0.17    ready   
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=coredns                                          
	                                                            k8s:io.kubernetes.pod.namespace=kube-system                                              
	                                                            k8s:k8s-app=kube-dns                                                                     
	 2459       Enabled            Enabled           14346      k8s:app=prometheus                                       fd02::5e   10.0.0.103   ready   
	                                                            k8s:io.cilium.k8s.policy.cluster=default                                                 
	                                                            k8s:io.cilium.k8s.policy.serviceaccount=prometheus-k8s                                   
	                                                            k8s:io.kubernetes.pod.namespace=cilium-monitoring                                        
	 
Stderr:
 	 

===================== Exiting AfterFailed =====================
01:56:39 STEP: Running AfterEach for block EntireTestsuite

[[ATTACHMENT|e1e3e85c_K8sPolicyTest_Clusterwide_policies_Test_clusterwide_connectivity_with_policies.zip]]
01:56:41 STEP: Running AfterAll block for EntireTestsuite K8sPolicyTest Clusterwide policies
01:56:41 STEP: Deleting namespace 2022081201531k8spolicytestclusterwidepoliciestestclusterwidecon
01:56:52 STEP: Deleting namespace 2022081201532k8spolicytestclusterwidepoliciestestclusterwidecon

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2323/artifact/4e9d440f_K8sFQDNTest_Validate_that_FQDN_policy_continues_to_work_after_being_updated.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2323/artifact/6dfdaedd_K8sFQDNTest_Restart_Cilium_validate_that_FQDN_is_still_working.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2323/artifact/a833270b_K8sFQDNTest_Validate_that_multiple_specs_are_working_correctly.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2323/artifact/e1e3e85c_K8sPolicyTest_Clusterwide_policies_Test_clusterwide_connectivity_with_policies.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9//2323/artifact/test_results_Cilium-PR-K8s-1.16-kernel-4.9_2323_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9/2323/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

Merged
6 tasks
@github-actions
Copy link

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Oct 11, 2022
@sayboras sayboras added pinned These issues are not marked stale by our issue bot. and removed stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. labels Oct 11, 2022
@pchaigno pchaigno removed the pinned These issues are not marked stale by our issue bot. label Apr 21, 2023
@pchaigno
Copy link
Member

Unpinning since it looks like this flake issue may be stale (not flaky anymore).

@sayboras sayboras mentioned this issue Apr 27, 2023
Merged
2 tasks
@github-actions
Copy link

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

@github-actions github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Jun 21, 2023
@github-actions
Copy link

github-actions bot commented Jul 5, 2023

This issue has not seen any activity since it was marked stale.
Closing.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ci/flake This is a known failure that occurs in the tree. Please investigate me! stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pchaigno @sayboras