Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite} #30566

Closed
k8s-github-robot opened this issue Aug 13, 2016 · 63 comments
Assignees
Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.

Comments

@k8s-github-robot
Copy link

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30564/node-pull-build-e2e-test/18635/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 13 00:56:04.176: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 22 (23),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30564.

@k8s-github-robot k8s-github-robot added priority/backlog Higher priority than priority/awaiting-more-evidence. kind/flake Categorizes issue or PR as related to a flaky test. labels Aug 13, 2016
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30500/node-pull-build-e2e-test/19050/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 16 08:50:09.959: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 23 (25),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30500.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30721/node-pull-build-e2e-test/19435/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 17 17:30:39.999: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 24 (26),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30721.

@k8s-github-robot k8s-github-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed priority/backlog Higher priority than priority/awaiting-more-evidence. labels Aug 17, 2016
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/8487/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 19 09:00:10.957: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 25 (26),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

@k8s-github-robot k8s-github-robot added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. and removed priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Aug 19, 2016
@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30707/node-pull-build-e2e-test/20172/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 19 21:39:18.017: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 19 (22),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30707.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30890/node-pull-build-e2e-test/20856/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 23 13:23:57.296: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 29 (30),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30890.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30091/node-pull-build-e2e-test/21362/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 25 19:22:24.461: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 26 (28),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30091.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/31491/node-pull-build-e2e-test/21573/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 26 19:40:35.448: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 28 (29),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #31491.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30932/node-pull-build-e2e-test/21665/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 28 17:32:24.707: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 16 (19),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30932.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/31471/node-pull-build-e2e-test/21666/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 28 18:44:24.943: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 25 (27),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #31471.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention...

@bprashanth
Copy link
Contributor

A majority of these seem to be a failure to watch from the given resource version: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/common/pods.go#L178, and all of them appear to be failures in node-pull-build.

The test lists pods, starts a watch on the RV in the pod list, creates a new pod, then waits for the creted pod on the watch. Unfortunately it doesn't handle any watch event except "Add". We should just modify the watch to use: watch.Until instead, eg: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/common/init_container.go#L84

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/31800/node-pull-build-e2e-test/22164/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Aug 31 17:54:53.479: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 23 (24),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #31800.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/8981/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Sep  3 18:36:57.441: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 21 (23),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

@davidopp
Copy link
Member

davidopp commented Sep 4, 2016

@bprashanth is #32040 what you had in mind?

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/30787/node-pull-build-e2e-test/22770/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Sep  5 23:56:27.023: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 24 (27),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #30787.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/33629/node-pull-build-e2e-test/26567/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Sep 29 23:28:46.672: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 19 (20),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #33629.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

2 similar comments
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/34323/node-pull-build-e2e-test/27498/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Oct  7 12:29:10.522: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 19 (20),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #34323.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

1 similar comment
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/34019/node-pull-build-e2e-test/27890/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Failed to observe pod deletion
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:255

Happened on a presubmit run in #34019.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/10141/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Oct 11 17:36:18.991: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 23 (27),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/34766/node-pull-build-e2e-test/28440/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:266
Oct 13 22:24:21.059: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 23 (26),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/node-pull-build-e2e-test/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:199

Happened on a presubmit run in #34766.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

6 similar comments
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/kubelet-gce-e2e-ci/10619/

Failed: [k8s.io] Pods should be submitted and removed [Conformance] {E2eNode Suite}

/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:261
Oct 26 23:25:24.392: Failed to observe pod creation: {ERROR &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,},Status:Failure,Message:too old resource version: 31 (33),Reason:Gone,Details:nil,Code:410,}}
/var/lib/jenkins/workspace/kubelet-gce-e2e-ci/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/common/pods.go:194

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

3 similar comments
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@calebamiles
Copy link
Contributor

@saad-ali we might be able to close this issue, the last flake from testgrid was 26 October 2016 and we've had solid green runs after that. What do you think @davidopp

@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

1 similar comment
@k8s-github-robot
Copy link
Author

[FLAKE-PING] @davidopp

This flaky-test issue would love to have more attention.

@saad-ali
Copy link
Member

saad-ali commented Nov 7, 2016

@saad-ali we might be able to close this issue, the last flake from testgrid was 26 October 2016 and we've had solid green runs after that. What do you think @davidopp

Ack, it is significantly less flaky now. Closing. It will be reopened if it flakes again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/flake Categorizes issue or PR as related to a flaky test. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Development

No branches or pull requests

6 participants