Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure cluster [6d268a...] failed 49 builds, 3 jobs, and 1 tests over 1 days #45998

Closed
k8s-github-robot opened this issue May 17, 2017 · 9 comments · Fixed by kubernetes/test-infra#4135
Closed

Failure cluster [6d268a...] failed 49 builds, 3 jobs, and 1 tests over 1 days #45998

k8s-github-robot opened this issue May 17, 2017 · 9 comments · Fixed by kubernetes/test-infra#4135
Assignees
@piosz
Labels
kind/flake Categorizes issue or PR as related to a flaky test.

Comments

@k8s-github-robot
Copy link

Failure cluster 6d268a83c25eb70f6b68

Error text:
error during ./hack/ginkgo-e2e.sh --ginkgo.focus=\[Feature:StackdriverLogging\]: exit status 1
Failure cluster statistics:

1 tests failed, 3 jobs failed, 49 builds failed.
Failure stats cover 1 day time range '15 May 2017 22:03 UTC' to '16 May 2017 22:03 UTC'.

Top failed tests by jobs failed:
Test Name Jobs Failed
Test 3
Top failed jobs by builds failed:
Job Name Builds Failed Latest Failure
ci-kubernetes-e2e-gke-stackdriver 44 16 May 2017 21:16 UTC
ci-kubernetes-e2e-gci-gce-sd-logging 3 16 May 2017 08:11 UTC
ci-kubernetes-e2e-gce-sd-logging 2 16 May 2017 11:40 UTC

Current Status
@k8s-github-robot k8s-github-robot added the kind/flake Categorizes issue or PR as related to a flaky test. label May 17, 2017
@jagosan
Copy link
Contributor

jagosan commented May 19, 2017

@fejta - trying to triage issues; tough to route to a sig with info included here. any suggestions?

@fejta
Copy link
Contributor

fejta commented May 19, 2017

ci-kubernetes-e2e-gke-stackdriver 44 16 May 2017 21:16 UTC

And if you open that link: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-stackdriver/3608

Pretty clear that this has to do with Feature:StackdriverMonitoring... any idea what sig cares about this feature and/or who is working on it?

@fejta
Copy link
Contributor

fejta commented May 19, 2017

Also if you search for Feature:StackdriverLogging:
https://github.com/kubernetes/kubernetes/search?utf8=%E2%9C%93&q=Feature%3AStackdriverLogging&type=

and then look at the history:
https://github.com/kubernetes/kubernetes/commits/a2f40cafcf726b504d6c2e15e25cab248827c779/test/e2e/cluster-logging/sd_load.go

So my guess is maybe:
/sig cluster-ops
/assign @crassirostris

@crassirostris
Copy link

@fejta Thanks! I'll take a look on Monday

Btw, what is the status with the test_owners.csv? I could've been tracked using this spreadsheet

@fejta
Copy link
Contributor

fejta commented May 19, 2017

Thanks much!

We haven't been forcing it to stay up to date because it causes to many merge conflicts, we're working on the replacement (see test_owners.json)

@crassirostris
Copy link

@fejta Cool, nice to hear! I actually liked when issues were auto-assigned to me back in the days

@crassirostris
Copy link

@fejta It's actually strange that those tests were merged, they run in different packages even

As for monitoring ones, I don't know the reason, but I'll try to find someone to track it. As for logging, I believe #45255 will fix it, once I'll figure out the godeps problem

@crassirostris crassirostris mentioned this issue May 22, 2017
Merged
@crassirostris
Copy link

But meanwhile I'll make logging tests less restrictive

k8s-github-robot pushed a commit that referenced this issue May 22, 2017
Merge pull request #46229 from crassirostris/sd-logging-e2e-less-rest…
c717a0e 
…rictive

Automatic merge from submit-queue

Make Stackdriver Logging e2e tests less restrictive

To reduce flakiness, as described in #45998, until further investigation

/cc @fgrzadkowski
@crassirostris crassirostris assigned piosz and unassigned crassirostris Aug 4, 2017
@piosz
Copy link
Member

piosz commented Aug 4, 2017

@fejta I fixed tests for gce. The only missing piece for gke is whitelisting the project on SD side.

@piosz piosz mentioned this issue Aug 21, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@piosz piosz

Labels
kind/flake Categorizes issue or PR as related to a flaky test.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Do not set STACKDRIVER_API_ENDPOINT_OVERRIDE flag on GKE piosz/test-infra
6 participants
@jagosan @fejta @crassirostris @piosz @k8s-github-robot @k8s-ci-robot