Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

node-e2e-containerd fails with image notFound error #85040

Closed
tedyu opened this issue Nov 9, 2019 · 11 comments
Closed

node-e2e-containerd fails with image notFound error #85040

tedyu opened this issue Nov 9, 2019 · 11 comments
Assignees
@Random-Liu
Labels
kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@tedyu
Copy link
Contributor

tedyu commented Nov 9, 2019

Which jobs are failing:
pull-kubernetes-node-e2e-containerd

Since when has it been failing:
Nov 9th, 2019

Testgrid link:
https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/84951/pull-kubernetes-node-e2e-containerd/1193222671123353600/build-log.txt

Reason for failure:

I1109 17:55:49.120] Failure Finished Test Suite on Host 
I1109 17:55:49.121] unable to create gce instance with running docker daemon for image cos-73-11647-348-0.  googleapi: Error 404: The resource 'projects/k8s-c8d-pr-node-e2e/zones/us-west1-b/instances/tmp-node-e2e-99fe64c3-cos-73-11647-348-0' was not found, notFound
I1109 17:55:49.121] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
I1109 17:55:49.121] <                              FINISH TEST                               <
I1109 17:55:49.121] <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
I1109 17:55:49.121] 
W1109 17:55:49.222] E1109 17:55:49.111434    4748 run_remote.go:747] Error deleting instance "tmp-node-e2e-99fe64c3-cos-73-11647-348-0": googleapi: Error 404: The resource 'projects/k8s-c8d-pr-node-e2e/zones/us-west1-b/instances/tmp-node-e2e-99fe64c3-cos-73-11647-348-0' was not found, notFound
I1109 17:55:49.783] 
I1109 17:55:49.784] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
I1109 17:55:49.785] >                              START TEST                                >
I1109 17:55:49.785] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
I1109 17:55:49.785] Start Test Suite on Host 
I1109 17:55:49.786] 
I1109 17:55:49.786] Failure Finished Test Suite on Host 
I1109 17:55:49.787] unable to create gce instance with running docker daemon for image ubuntu-gke-1804-d1809-0-v20191107.  googleapi: Error 404: The resource 'projects/k8s-c8d-pr-node-e2e/zones/us-west1-b/instances/tmp-node-e2e-99fe64c3-ubuntu-gke-1804-d1809-0-v20191107' was not found, notFound

Anything else we need to know:
@tedyu tedyu added the kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. label Nov 9, 2019
@k8s-ci-robot k8s-ci-robot added the needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. label Nov 9, 2019
@neolit123
Copy link
Member

/sig node

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 9, 2019
@alculquicondor
Copy link
Member

/sig testing

@k8s-ci-robot k8s-ci-robot added the sig/testing Categorizes an issue or PR as relevant to SIG Testing. label Nov 12, 2019
@alculquicondor
Copy link
Member

/remove-sig testing

@k8s-ci-robot k8s-ci-robot removed the sig/testing Categorizes an issue or PR as relevant to SIG Testing. label Nov 12, 2019
@dchen1107
Copy link
Member

/assign @Random-Liu

This was referenced Nov 13, 2019
Merged
Merged
@chuckha chuckha mentioned this issue Nov 13, 2019
Merged
This was referenced Nov 13, 2019
Merged
Merged
Merged
@RainbowMango RainbowMango mentioned this issue Nov 14, 2019
Merged
@RainbowMango
Copy link
Member

Who is handling this issue?

This was referenced Nov 14, 2019
Merged
Merged
@hoegaarden hoegaarden mentioned this issue Nov 14, 2019
Closed
@busser busser mentioned this issue Nov 15, 2019
Closed
@josiahbjorgaard
Copy link
Contributor

Flakes related to this issue are currently blocking PRs merging for v1.17.

dims added a commit to dims/test-infra that referenced this issue Nov 15, 2019
@dims
Do not run pull-kubernetes-node-e2e-containerd always
c01c45e 
There is no point in running this always ... until it is fixed

kubernetes/kubernetes#85303
kubernetes/kubernetes#85040
@dims dims mentioned this issue Nov 15, 2019
Closed
@Random-Liu
Copy link
Member

This is a quota issue.

The GCE project doesn't have too many quota.

I'll address it.

@Random-Liu
Copy link
Member

Basically we have limited IP quota in that presubmit project, and too many changes before code freeze and hit the quota limit. :P

@Random-Liu
Copy link
Member

Random-Liu commented Nov 15, 2019 •
edited

Sent out the quota increase request and deleted VMs leaked by some timeout test.

Let's see whether it fixes the issue.

@dims
Copy link
Member

dims commented Nov 16, 2019

thanks @Random-Liu

/close

@k8s-ci-robot
Copy link
Contributor

@dims: Closing this issue.

In response to this:

thanks @Random-Liu

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@odinuge odinuge mentioned this issue Jan 9, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Random-Liu Random-Liu

Labels
kind/failing-test Categorizes issue or PR as related to a consistently or frequently failing test. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@dims @tedyu @neolit123 @alculquicondor @Random-Liu @josiahbjorgaard @dchen1107 @RainbowMango @k8s-ci-robot