Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: gke-stable #17857

Closed
Weil0ng opened this issue Nov 11, 2021 · 1 comment
Closed

CI: gke-stable #17857

Weil0ng opened this issue Nov 11, 2021 · 1 comment
Labels
area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me!

Comments

@Weil0ng
Copy link
Contributor

Weil0ng commented Nov 11, 2021

CI failure

gke-stable fails in https://jenkins.cilium.io/job/Cilium-PR-K8s-GKE/6900/ due to one of the test pods fail health check, seems like a flake?

14:40:09  22:40:08 STEP: WaitforPods(namespace="default", filter="-l zgroup=test-k8s2") => timed out waiting for pods with filter -l zgroup=test-k8s2 to be ready: 4m0s timeout expired
14:40:09  22:40:09 STEP: cmd: kubectl describe pods -n default -l zgroup=test-k8s2
14:40:09  Exitcode: 0 
14:40:09  Stdout:
14:40:09   	 Name:         test-k8s2-79ff876c9d-r4pnc
14:40:09  	 Namespace:    default
14:40:09  	 Priority:     0
14:40:09  	 Node:         gke-cilium-ci-10-cilium-ci-10-f656816e-cm7x/10.128.15.222
14:40:09  	 Start Time:   Wed, 10 Nov 2021 22:35:51 +0000
14:40:09  	 Labels:       pod-template-hash=79ff876c9d
14:40:09  	               zgroup=test-k8s2
14:40:09  	 Annotations:  <none>
14:40:09  	 Status:       Running
14:40:09  	 IP:           10.48.1.1
14:40:09  	 IPs:
14:40:09  	   IP:           10.48.1.1
14:40:09  	 Controlled By:  ReplicaSet/test-k8s2-79ff876c9d
14:40:09  	 Containers:
14:40:09  	   web:
14:40:09  	     Container ID:   containerd://2e1cf68e14e6853986326c358a8403306083a30dfbc94a2bf7892d0b7d58315e
14:40:09  	     Image:          docker.io/cilium/echoserver:1.10.1
14:40:09  	     Image ID:       docker.io/cilium/echoserver@sha256:daa2b422465b8714195fda33bc14865a8672c14963d0955c9d95632d6b804b63
14:40:09  	     Port:           80/TCP
14:40:09  	     Host Port:      8080/TCP
14:40:09  	     State:          Running
14:40:09  	       Started:      Wed, 10 Nov 2021 22:35:58 +0000
14:40:09  	     Ready:          False
14:40:09  	     Restart Count:  0
14:40:09  	     Readiness:      http-get http://:80/ delay=0s timeout=1s period=10s #success=1 #failure=3
14:40:09  	     Environment:    <none>
14:40:09  	     Mounts:
14:40:09  	       /var/run/secrets/kubernetes.io/serviceaccount from default-token-tc7rj (ro)
14:40:09  	   udp:
14:40:09  	     Container ID:   containerd://d038b7f739f1bc4e760f78d0be57b58599690f821569c997da408de3e4ccfbe8
14:40:09  	     Image:          docker.io/cilium/echoserver-udp:v2020.01.30
14:40:09  	     Image ID:       docker.io/cilium/echoserver-udp@sha256:66680075424a25cae98c4a437fec2b7e33b91fbe94e53defeea0305aee3866dd
14:40:09  	     Port:           69/UDP
14:40:09  	     Host Port:      6969/UDP
14:40:09  	     State:          Running
14:40:09  	       Started:      Wed, 10 Nov 2021 22:35:58 +0000
14:40:09  	     Ready:          True
14:40:09  	     Restart Count:  0
14:40:09  	     Environment:    <none>
14:40:09  	     Mounts:
14:40:09  	       /var/run/secrets/kubernetes.io/serviceaccount from default-token-tc7rj (ro)
14:40:09  	 Conditions:
14:40:09  	   Type              Status
14:40:09  	   Initialized       True 
14:40:09  	   Ready             False 
14:40:09  	   ContainersReady   False 
14:40:09  	   PodScheduled      True 
14:40:09  	 Volumes:
14:40:09  	   default-token-tc7rj:
14:40:09  	     Type:        Secret (a volume populated by a Secret)
14:40:09  	     SecretName:  default-token-tc7rj
14:40:09  	     Optional:    false
14:40:09  	 QoS Class:       BestEffort
14:40:09  	 Node-Selectors:  cilium.io/ci-node=k8s2
14:40:09  	 Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
14:40:09  	                  node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
14:40:09  	 Events:
14:40:09  	   Type     Reason     Age                  From               Message
14:40:09  	   ----     ------     ----                 ----               -------
14:40:09  	   Normal   Scheduled  4m18s                default-scheduler  Successfully assigned default/test-k8s2-79ff876c9d-r4pnc to gke-cilium-ci-10-cilium-ci-10-f656816e-cm7x
14:40:09  	   Normal   Pulling    4m16s                kubelet            Pulling image "docker.io/cilium/echoserver:1.10.1"
14:40:09  	   Normal   Pulled     4m11s                kubelet            Successfully pulled image "docker.io/cilium/echoserver:1.10.1" in 4.818380138s
14:40:09  	   Normal   Created    4m11s                kubelet            Created container web
14:40:09  	   Normal   Started    4m11s                kubelet            Started container web
14:40:09  	   Normal   Pulled     4m11s                kubelet            Container image "docker.io/cilium/echoserver-udp:v2020.01.30" already present on machine
14:40:09  	   Normal   Created    4m11s                kubelet            Created container udp
14:40:09  	   Normal   Started    4m11s                kubelet            Started container udp
14:40:09  	   Warning  Unhealthy  78s (x18 over 4m8s)  kubelet            Readiness probe failed: Get "http://10.48.1.1:80/": dial tcp 10.48.1.1:80: connect: connection refused
14:40:09  	 
14:40:09  Stderr:
14:40:09   	 
14:40:09  
14:40:09  FAIL: Expected
14:40:09      <*errors.errorString | 0xc000696730>: {
14:40:09          s: "timed out waiting for pods with filter -l zgroup=test-k8s2 to be ready: 4m0s timeout expired",
14:40:09      }
14:40:09  to be nil
@Weil0ng Weil0ng added area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me! labels Nov 11, 2021
@pchaigno
Copy link
Member

Duplicate of #17307.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/CI Continuous Integration testing issue or flake ci/flake This is a known failure that occurs in the tree. Please investigate me!
Projects
None yet
Development

No branches or pull requests

2 participants