Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kubernetes-soak-continuous-e2e-gke-gci: broken test run #34868

Closed
k8s-github-robot opened this issue Oct 14, 2016 · 124 comments
Closed

kubernetes-soak-continuous-e2e-gke-gci: broken test run #34868

k8s-github-robot opened this issue Oct 14, 2016 · 124 comments
Assignees
@krousey
Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone
v1.5

Comments

@k8s-github-robot
Copy link

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/212/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 14 13:39:08.813: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30260
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 14 15:36:59.483: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 177893376
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 203177984
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 130281472
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 14 16:01:51.798: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31345/hostName
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 14 15:55:11.912: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30642/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 14 15:44:25.443: Could not reach HTTP service through 162.222.177.33:31414 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 14 12:49:40.276: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.142:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30702&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 14 12:00:43.513: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 115798016
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 161579008
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 179879936
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 14 15:01:43.746: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 196362240, container "runtime": expected RSS memory (MB) < 314572800; got 334512128
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 216797184, container "runtime": expected RSS memory (MB) < 314572800; got 328830976
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 148377600, container "runtime": expected RSS memory (MB) < 314572800; got 329064448
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 14 11:16:10.823: Could not reach HTTP service through 162.222.177.33:32169 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 14 13:35:51.155: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.194:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32504&tries=1'
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 14 11:32:25.830: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.155:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=32063&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 14 14:37:13.152: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30806
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 14 10:46:16.911: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.130:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=30221&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Previous issues for this suite: #34202 #34689 #34799
@k8s-github-robot k8s-github-robot added kind/flake Categorizes issue or PR as related to a flaky test. priority/backlog Higher priority than priority/awaiting-more-evidence. area/test-infra labels Oct 14, 2016
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/213/

Multiple broken tests:

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 14 21:04:39.254: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 154013696
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 174981120
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 100786176
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 14 16:52:10.400: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31841
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 14 17:09:42.018: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.55:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31925&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 14 17:37:25.543: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.84:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=30221&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 14 17:48:05.210: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.89:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30067&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 14 20:17:14.684: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 182435840, container "runtime": expected RSS memory (MB) < 314572800; got 338677760
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 202375168, container "runtime": expected RSS memory (MB) < 314572800; got 333340672
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 125333504, container "runtime": expected RSS memory (MB) < 314572800; got 330121216
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 14 17:03:53.576: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30552/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 14 19:21:40.598: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 159027200
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 182431744
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 105816064
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 14 22:13:53.813: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31080/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 14 21:17:45.924: Could not reach HTTP service through 162.222.177.33:32739 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 14 19:39:24.296: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.108:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31568&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 14 22:23:00.259: Could not reach HTTP service through 162.222.177.33:31589 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 14 21:30:21.579: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31971
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/214/

Multiple broken tests:

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 15 00:29:28.982: Could not reach HTTP service through 162.222.177.33:30947 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 04:23:42.533: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 198717440
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 121397248
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 173654016
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 14 23:04:47.673: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.26:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31285&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 02:49:46.887: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 213438464, container "runtime": expected RSS memory (MB) < 314572800; got 336420864
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 136359936, container "runtime": expected RSS memory (MB) < 314572800; got 335523840
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 191582208, container "runtime": expected RSS memory (MB) < 314572800; got 343605248
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 14 23:00:00.498: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32613/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] kubelet [k8s.io] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:145
Oct 14 23:36:19.505: Couldn't delete ns: "e2e-tests-kubelet-u2g9k": the server cannot complete the requested operation at this time, try again later (&errors.StatusError{ErrStatus:unversioned.Status{TypeMeta:unversioned.TypeMeta{Kind:"", APIVersion:""}, ListMeta:unversioned.ListMeta{SelfLink:"", ResourceVersion:""}, Status:"Failure", Message:"the server cannot complete the requested operation at this time, try again later", Reason:"ServerTimeout", Details:(*unversioned.StatusDetails)(0xc82147b0e0), Code:504}})
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:359

Issues about this test specifically: #28106

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 14 23:20:21.148: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31540
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 15 01:57:31.074: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.146:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=30211&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 15 03:26:47.812: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.27:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31095&tries=1'
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 14 22:53:51.727: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30996
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 14 23:42:46.444: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.231:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32530&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 02:22:36.973: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 183074816
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 108613632
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 162000896
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 14 22:51:20.678: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32595/hostName
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 15 01:39:07.680: Could not reach HTTP service through 162.222.177.33:30781 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/215/

Multiple broken tests:

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 15 06:43:59.283: Could not reach HTTP service through 162.222.177.33:30159 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 07:45:32.426: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 179892224
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 205299712
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 127709184
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 15 07:50:18.218: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30122/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 15 08:11:33.300: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32498
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 15 10:14:14.267: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.34:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32566&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 15 10:47:55.375: Could not reach HTTP service through 162.222.177.33:32118 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 15 09:40:14.212: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30603
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 15 09:06:43.867: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.47:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=32598&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 07:18:05.711: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 193351680
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 116854784
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 171393024
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 15 05:06:41.985: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31559/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 15 10:41:49.989: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.98:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32139&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 15 08:52:06.223: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.133:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=32418&tries=1'
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 06:16:53.457: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 141570048, container "runtime": expected RSS memory (MB) < 314572800; got 338935808
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 193896448, container "runtime": expected RSS memory (MB) < 314572800; got 344854528
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 223698944, container "runtime": expected RSS memory (MB) < 314572800; got 338030592
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/216/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 15 15:35:54.902: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.33:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30322&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 16:04:25.554: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 195850240, container "runtime": expected RSS memory (MB) < 314572800; got 349110272
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "runtime": expected RSS memory (MB) < 314572800; got 340660224, container "kubelet": expected RSS memory (MB) < 83886080; got 243027968
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 132018176, container "runtime": expected RSS memory (MB) < 314572800; got 341282816
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 12:35:52.632: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 162160640
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 214278144
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 107204608
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 15 11:38:57.027: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.115:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32157&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 15 15:14:34.760: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30804/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 15 15:30:56.460: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.30:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=32582&tries=1'
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 11:19:46.282: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 198529024
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 97058816
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 152154112
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 15 14:29:45.188: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.117:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31865&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 15 16:24:02.135: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30864
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 15 12:38:56.728: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32309
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 15 12:54:53.900: Could not reach HTTP service through 162.222.177.33:30065 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 15 12:07:30.778: Could not reach HTTP service through 162.222.177.33:31844 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 15 12:13:53.819: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31696/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/217/

Multiple broken tests:

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 20:42:01.875: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 204357632, container "runtime": expected RSS memory (MB) < 314572800; got 350879744
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 255262720, container "runtime": expected RSS memory (MB) < 314572800; got 343396352
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 136720384, container "runtime": expected RSS memory (MB) < 314572800; got 344788992
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 15 17:29:33.187: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32279/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 15 21:01:27.137: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32481/hostName
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 15 18:05:25.793: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.222:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32455&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 15 19:34:33.566: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.174:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31501&tries=1'
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 15 18:11:53.850: Could not reach HTTP service through 162.222.177.33:32205 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 15 20:17:07.628: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32558
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 22:38:53.904: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 237576192
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 124174336
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 178237440
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 15 20:48:31.918: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31392
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 15 19:43:24.191: Could not reach HTTP service through 162.222.177.33:32000 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 15 19:55:15.182: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.182:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31116&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 15 22:48:49.004: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.118:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30471&tries=1'
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 17:52:12.602: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 107319296
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 162586624
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 212533248
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/218/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 15 20:48:31.918: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31392
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 17:52:12.602: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 107319296
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 162586624
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 212533248
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 15 18:05:25.793: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.222:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32455&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 15 19:43:24.191: Could not reach HTTP service through 162.222.177.33:32000 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 15 21:01:27.137: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32481/hostName
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 15 22:48:49.004: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.118:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30471&tries=1'
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 20:42:01.875: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 204357632, container "runtime": expected RSS memory (MB) < 314572800; got 350879744
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 255262720, container "runtime": expected RSS memory (MB) < 314572800; got 343396352
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 136720384, container "runtime": expected RSS memory (MB) < 314572800; got 344788992
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 15 22:38:53.904: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 237576192
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 124174336
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 178237440
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 15 17:29:33.187: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32279/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 15 19:34:33.566: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.174:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31501&tries=1'
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 15 18:11:53.850: Could not reach HTTP service through 162.222.177.33:32205 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 15 19:55:15.182: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.182:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31116&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 15 20:17:07.628: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32558
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/219/

Multiple broken tests:

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 16 14:08:04.948: Could not reach HTTP service through 162.222.177.33:32613 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 11:05:52.617: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 163446784, container "runtime": expected RSS memory (MB) < 314572800; got 355155968
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 211120128, container "runtime": expected RSS memory (MB) < 314572800; got 346718208
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 135442432, container "runtime": expected RSS memory (MB) < 314572800; got 347725824
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 16 10:39:54.208: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30662/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 16 11:19:19.933: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.135:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31772&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 16 12:35:03.568: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.26:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31503&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 16 13:42:47.388: Could not reach HTTP service through 162.222.177.33:32405 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 11:39:57.039: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 136863744
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 167211008
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 109060096
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Proxy version v1 should proxy logs on node [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/proxy.go:64
Expected error:
    <*errors.StatusError | 0xc82141b080>: {
        ErrStatus: {
            TypeMeta: {Kind: "", APIVersion: ""},
            ListMeta: {SelfLink: "", ResourceVersion: ""},
            Status: "Failure",
            Message: "an error on the server (\"Error: 'EOF'\\nTrying to reach: 'https://10.240.0.4:10250/logs/'\") has prevented the request from succeeding",
            Reason: "InternalError",
            Details: {
                Name: "",
                Group: "",
                Kind: "",
                Causes: [
                    {
                        Type: "UnexpectedServerResponse",
                        Message: "Error: 'EOF'\nTrying to reach: 'https://10.240.0.4:10250/logs/'",
                        Field: "",
                    },
                ],
                RetryAfterSeconds: 0,
            },
            Code: 503,
        },
    }
    an error on the server ("Error: 'EOF'\nTrying to reach: 'https://10.240.0.4:10250/logs/'") has prevented the request from succeeding
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/proxy.go:332

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 09:44:06.502: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 174546944
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 114442240
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 139235328
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 16 13:48:45.819: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30064
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 16 09:55:19.709: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.184:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30197&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 16 10:01:13.417: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32331
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 16 14:18:29.303: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.92:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32548&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 16 12:55:19.106: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32403/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/220/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 16 16:03:26.573: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32181/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 16 16:09:44.403: Could not reach HTTP service through 162.222.177.33:32279 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 18:58:02.357: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 176205824, container "runtime": expected RSS memory (MB) < 314572800; got 359972864
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 210046976, container "runtime": expected RSS memory (MB) < 314572800; got 352735232
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 147894272, container "runtime": expected RSS memory (MB) < 314572800; got 352919552
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 16 16:11:38.134: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31876/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 16 16:24:35.394: Could not reach HTTP service through 162.222.177.33:30450 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 16 19:33:47.724: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.85:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31500&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 15:43:01.172: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 150036480
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 183144448
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 125329408
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 17:21:20.895: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 144879616
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 175247360
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 118079488
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 16 19:22:44.892: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.192:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32370&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 16 15:08:38.504: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30823
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 16 17:25:44.326: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.41:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30351&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 16 18:24:25.776: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31907
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 16 16:18:43.433: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.164:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31680&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/221/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 17 01:30:44.040: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31230
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 17 02:40:16.700: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31109
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 16 21:31:33.453: Could not reach HTTP service through 162.222.177.33:31861 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 16 22:01:45.674: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.140:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31467&tries=1'
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 17 02:42:12.711: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31684/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 17 00:56:41.956: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 157495296, container "runtime": expected RSS memory (MB) < 131072000; got 132526080
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 186916864
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 104828928
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 16 21:38:42.584: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.128:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31113&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 23:49:45.391: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 192917504, container "runtime": expected RSS memory (MB) < 314572800; got 361631744
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 218259456, container "runtime": expected RSS memory (MB) < 314572800; got 352260096
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 129179648, container "runtime": expected RSS memory (MB) < 314572800; got 353157120
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 16 22:07:41.676: Could not reach HTTP service through 162.222.177.33:32632 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 23:24:06.931: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 194191360
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 110211072
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 161280000, container "runtime": expected RSS memory (MB) < 209715200; got 211648512
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 17 01:41:42.874: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.141:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32247&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 17 01:37:20.615: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30550/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 16 22:29:50.272: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.173:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32399&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/222/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 16 21:38:42.584: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.128:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31113&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 17 01:30:44.040: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31230
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 17 02:42:12.711: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31684/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 23:24:06.931: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 194191360
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 110211072
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 161280000, container "runtime": expected RSS memory (MB) < 209715200; got 211648512
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 17 01:37:20.615: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30550/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 16 23:49:45.391: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 83886080; got 192917504, container "runtime": expected RSS memory (MB) < 314572800; got 361631744
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 218259456, container "runtime": expected RSS memory (MB) < 314572800; got 352260096
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 129179648, container "runtime": expected RSS memory (MB) < 314572800; got 353157120
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 16 22:01:45.674: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.140:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31467&tries=1'
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 17 02:40:16.700: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31109
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 17 01:41:42.874: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.141:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32247&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 16 22:07:41.676: Could not reach HTTP service through 162.222.177.33:32632 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 17 00:56:41.956: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 157495296, container "runtime": expected RSS memory (MB) < 131072000; got 132526080
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 186916864
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 104828928
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 16 21:31:33.453: Could not reach HTTP service through 162.222.177.33:31861 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 16 22:29:50.272: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.173:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=32399&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

@k8s-github-robot k8s-github-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Oct 17, 2016
@ixdy ixdy assigned spxtr and unassigned ixdy Oct 17, 2016
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/223/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 17 13:48:28.527: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.160:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31876&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 17 15:21:12.553: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.97:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=30074&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 17 13:36:19.531: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32347
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 17 13:38:26.633: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 30609
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 17 14:38:22.474: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "runtime": expected RSS memory (MB) < 314572800; got 365035520, container "kubelet": expected RSS memory (MB) < 83886080; got 154587136
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 184049664, container "runtime": expected RSS memory (MB) < 314572800; got 358731776
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "runtime": expected RSS memory (MB) < 314572800; got 357355520, container "kubelet": expected RSS memory (MB) < 83886080; got 127062016
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 17 17:19:23.416: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 137060352, container "runtime": expected RSS memory (MB) < 209715200; got 217415680
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 168542208
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "runtime": expected RSS memory (MB) < 209715200; got 210251776, container "kubelet": expected RSS memory (MB) < 73400320; got 112177152
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 17 14:46:53.409: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:30329/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 17 13:23:25.324: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.140:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=30580&tries=1'
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 17 15:07:02.301: Could not reach HTTP service through 162.222.177.33:31468 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 17 15:09:09.913: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31215/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 17 17:43:54.657: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 104206336
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 130596864, container "runtime": expected RSS memory (MB) < 131072000; got 136179712
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 160157696
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 17 18:49:36.174: Could not reach HTTP service through 162.222.177.33:32006 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 17 13:54:14.243: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.115:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31936&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/224/

Multiple broken tests:

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:105
Oct 17 21:32:13.725: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.38:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=31303&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34317

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:114
Oct 17 21:36:57.178: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:31392/hostName
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #32684

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for pod-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:96
Oct 17 19:52:05.775: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.0.35:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31128&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:141
Oct 18 00:51:50.183: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.2.25:8080/dial?request=hostName&protocol=udp&host=162.222.177.33&port=32508&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34064

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for endpoint-Service: http {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:132
Oct 18 00:58:29.687: Failed to find expected endpoints:
Tries 0
Command curl -q -s 'http://10.180.1.38:8080/dial?request=hostName&protocol=http&host=162.222.177.33&port=31320&tries=1'
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:206

Issues about this test specifically: #34104

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: udp [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:187
Oct 17 20:52:31.998: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 32711
retrieved map[]
expected map[netserver-1:{} netserver-2:{} netserver-0:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33285

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 18 00:47:22.755: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 147628032, container "runtime": expected RSS memory (MB) < 209715200; got 223248384
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "runtime": expected RSS memory (MB) < 209715200; got 213196800, container "kubelet": expected RSS memory (MB) < 73400320; got 180031488
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 126054400, container "runtime": expected RSS memory (MB) < 209715200; got 212733952
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Services should be able to create a functioning NodePort service {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:463
Oct 17 19:57:55.863: Could not reach HTTP service through 162.222.177.33:30376 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #28064 #28569 #34036

Failed: [k8s.io] Services should be able to change the type and ports of a service [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:782
Oct 17 20:17:34.532: Could not reach HTTP service through 162.222.177.33:31526 after 5m0s: timed out waiting for the condition
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/service.go:1994

Issues about this test specifically: #26134

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should function for node-Service: udp {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:123
Oct 17 22:20:57.810: Failed to find expected endpoints:
Tries 0
Command echo 'hostName' | timeout -t 3 nc -w 1 -u 162.222.177.33 31190
retrieved map[]
expected map[netserver-0:{} netserver-1:{} netserver-2:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 18 00:18:56.087: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "kubelet": expected RSS memory (MB) < 73400320; got 139341824, container "runtime": expected RSS memory (MB) < 131072000; got 138326016
node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 73400320; got 169525248
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 73400320; got 115728384
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:278
Oct 17 23:25:10.535: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-badc50ee-dck3:
 container "kubelet": expected RSS memory (MB) < 83886080; got 197517312, container "runtime": expected RSS memory (MB) < 314572800; got 362491904
node gke-jenkins-e2e-default-pool-badc50ee-ex79:
 container "kubelet": expected RSS memory (MB) < 83886080; got 139821056, container "runtime": expected RSS memory (MB) < 314572800; got 362192896
node gke-jenkins-e2e-default-pool-badc50ee-0va8:
 container "runtime": expected RSS memory (MB) < 314572800; got 372420608, container "kubelet": expected RSS memory (MB) < 83886080; got 167784448
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:153

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Networking [k8s.io] Granular Checks: Services [Slow] should update nodePort: http [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/networking.go:175
Oct 17 22:56:28.344: Failed to find expected endpoints:
Tries 0
Command curl -q -s --connect-timeout 1 http://162.222.177.33:32040/hostName
retrieved map[]
expected map[netserver-2:{} netserver-0:{} netserver-1:{}]

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/networking_utils.go:255

Issues about this test specifically: #33730

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/225/

Multiple broken tests:

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:217
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27115 #28070 #30747 #31341

Failed: [k8s.io] SchedulerPredicates [Serial] validates MaxPods limit number of pods that are allowed to run [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:143
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27662 #29820 #31971 #32505 #34221

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #26784 #28384 #31935 #33023

@spxtr spxtr assigned krousey and unassigned spxtr Oct 18, 2016
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/226/

Multiple broken tests:

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] SchedulerPredicates [Serial] validates MaxPods limit number of pods that are allowed to run [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:143
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27662 #29820 #31971 #32505 #34221

Failed: [k8s.io] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:217
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27115 #28070 #30747 #31341

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #26784 #28384 #31935 #33023

@krousey
Copy link
Contributor

krousey commented Oct 18, 2016

@yujuhong It seems that resource usage collection is now failing. I see a lot of lines like:

Oct 18 08:11:43.036: INFO: Missing info/stats for container "misc" on node "gke-jenkins-e2e-default-pool-8949a407-34zy"

in the build log. Is kubelet no longer reporting these?

@yujuhong
Copy link
Contributor

@krousey there is no "misc" container on GCI nodes. Since the tests don't actually check the resource usage for "misc", I just removed this in the PR yesterday #34991.
The soak cluster will continue to fail until it gets redeployed (next week?).

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/227/

Multiple broken tests:

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] SchedulerPredicates [Serial] validates MaxPods limit number of pods that are allowed to run [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:143
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27662 #29820 #31971 #32505 #34221

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Expected error:
    <errors.aggregate | len:3, cap:4>: [
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-ket9\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-34zy\" is not ready yet",
        },
        {
            s: "Resource usage on node \"gke-jenkins-e2e-default-pool-8949a407-idyl\" is not ready yet",
        },
    ]
    [Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-ket9" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-34zy" is not ready yet, Resource usage on node "gke-jenkins-e2e-default-pool-8949a407-idyl" is not ready yet]
not to have occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:105

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:217
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27115 #28070 #30747 #31341

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/230/

Multiple broken tests:

Failed: [k8s.io] SchedulerPredicates [Serial] validates MaxPods limit number of pods that are allowed to run [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:143
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27662 #29820 #31971 #32505 #34221 #35106 #35110 #35121

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Oct 19 08:40:26.670: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-f0421644-fv0f:
 container "kubelet": expected RSS memory (MB) < 73400320; got 77701120
node gke-jenkins-e2e-default-pool-f0421644-cv4y:
 container "kubelet": expected RSS memory (MB) < 73400320; got 79192064
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:154

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Oct 19 10:23:49.403: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-f0421644-cv4y:
 container "kubelet": expected RSS memory (MB) < 73400320; got 93036544
node gke-jenkins-e2e-default-pool-f0421644-dlec:
 container "kubelet": expected RSS memory (MB) < 73400320; got 85970944
node gke-jenkins-e2e-default-pool-f0421644-fv0f:
 container "kubelet": expected RSS memory (MB) < 73400320; got 91406336
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:154

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Oct 19 11:42:33.433: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-f0421644-cv4y:
 container "kubelet": expected RSS memory (MB) < 83886080; got 115269632
node gke-jenkins-e2e-default-pool-f0421644-dlec:
 container "kubelet": expected RSS memory (MB) < 83886080; got 111468544
node gke-jenkins-e2e-default-pool-f0421644-fv0f:
 container "kubelet": expected RSS memory (MB) < 83886080; got 116338688
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:154

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:217
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27115 #28070 #30747 #31341

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/231/

Multiple broken tests:

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 0 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Oct 19 15:28:08.155: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-f0421644-cv4y:
 container "kubelet": expected RSS memory (MB) < 73400320; got 109170688
node gke-jenkins-e2e-default-pool-f0421644-dlec:
 container "kubelet": expected RSS memory (MB) < 73400320; got 104071168
node gke-jenkins-e2e-default-pool-f0421644-fv0f:
 container "kubelet": expected RSS memory (MB) < 73400320; got 103661568
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:154

Issues about this test specifically: #26784 #28384 #31935 #33023

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 35 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Oct 19 13:55:07.930: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-f0421644-cv4y:
 container "kubelet": expected RSS memory (MB) < 73400320; got 112967680
node gke-jenkins-e2e-default-pool-f0421644-dlec:
 container "kubelet": expected RSS memory (MB) < 73400320; got 104255488
node gke-jenkins-e2e-default-pool-f0421644-fv0f:
 container "kubelet": expected RSS memory (MB) < 73400320; got 106778624
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:154

Issues about this test specifically: #28220 #32942

Failed: [k8s.io] Kubelet [Serial] [Slow] [k8s.io] regular resource usage tracking resource tracking for 100 pods per node {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:279
Oct 19 14:20:56.265: Memory usage exceeding limits:
 node gke-jenkins-e2e-default-pool-f0421644-cv4y:
 container "kubelet": expected RSS memory (MB) < 83886080; got 127889408
node gke-jenkins-e2e-default-pool-f0421644-dlec:
 container "kubelet": expected RSS memory (MB) < 83886080; got 124043264
node gke-jenkins-e2e-default-pool-f0421644-fv0f:
 container "kubelet": expected RSS memory (MB) < 83886080; got 128991232
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/kubelet_perf.go:154

Issues about this test specifically: #26982 #32214 #33994 #34035

Failed: [k8s.io] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:217
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27115 #28070 #30747 #31341

Failed: [k8s.io] SchedulerPredicates [Serial] validates MaxPods limit number of pods that are allowed to run [Slow] {Kubernetes e2e suite}

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:143
Not scheduled Pods: []api.Pod(nil)
Expected
    <int>: 0
to equal
    <int>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduler_predicates.go:933

Issues about this test specifically: #27662 #29820 #31971 #32505 #34221 #35106 #35110 #35121

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/414/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/415/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/416/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/417/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/418/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/419/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/420/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/421/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/422/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/423/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/424/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/425/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/426/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/427/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/428/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/429/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/430/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/431/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/432/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/433/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/434/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/435/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/436/

Run so broken it didn't make JUnit output!

@k8s-github-robot
Copy link
Author

Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubernetes-soak-continuous-e2e-gke-gci/437/

Run so broken it didn't make JUnit output!

@yujuhong
Copy link
Contributor

The soak cluster needs to be re-deployed to include the fix #36680.
/cc @k8s-oncall

@calebamiles calebamiles added this to the v1.5 milestone Nov 15, 2016
@dims
Copy link
Member

dims commented Nov 17, 2016

cc @kubernetes/test-infra-maintainers @kubernetes/test-infra-admins @kubernetes/sig-testing - Folks, what's the procedure to request an update of the soak cluster?

Please let us know so we can close out this issue as a release blocker or release non blocker for v1.5 release.

Thanks,
Dims

@fejta
Copy link
Contributor

fejta commented Nov 17, 2016

k8s-oncall should be able to do this. Redeployed cluster earlier today so closing this one out.

@fejta fejta closed this as completed Nov 17, 2016
@dims
Copy link
Member

dims commented Nov 17, 2016

thanks a ton @fejta

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@krousey krousey

Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
Milestone
v1.5
Development

No branches or pull requests
8 participants
@dims @krousey @calebamiles @fejta @spxtr @ixdy @yujuhong @k8s-github-robot