Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add debugging for aggregator flake #64587

Merged
merged 1 commit into from
Jun 1, 2018

Conversation

liggitt
Copy link
Member

@liggitt liggitt commented Jun 1, 2018

capture current state of the APIService and extension server pod for output in failure cases

xref #63622

NONE

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 1, 2018
@k8s-ci-robot k8s-ci-robot requested review from gmarek and ixdy June 1, 2018 00:16
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jun 1, 2018
@jennybuckley
Copy link

jennybuckley commented Jun 1, 2018

Thanks! this is really helpful, that test doesn't really output very much which is making it pretty hard to debug.

I opened #64587 to capture the state of the APIService, extension server pods, and those pod logs in failures cases

We might also need to get the logs from the pod that is running the sample-apiserver even if it doesn't fail. From the logs I saw it looked like it wasn't failing, just taking a long time (or forever) to start the server. Not sure how possible that is to do, though.

@liggitt
Copy link
Member Author

liggitt commented Jun 1, 2018

We might also need to get the logs from the pod that is running the sample-apiserver even if it doesn't fail. From the logs I saw it looked like it wasn't failing, just taking a long time (or forever) to start the server. Not sure how possible that is to do, though.

that would cause a timeout failure, which this should capture

@jennybuckley
Copy link

ah I see you were saying if the test failed, not if the pod failed. My mistake. Are you going to just keep retesting this until it flakes?

@liggitt
Copy link
Member Author

liggitt commented Jun 1, 2018

Are you going to just keep retesting this until it flakes?

since it only kicks in when the test fails, I'd probably merge this to get data from CI runs

@liggitt
Copy link
Member Author

liggitt commented Jun 1, 2018

/retest

@liggitt
Copy link
Member Author

liggitt commented Jun 1, 2018

/assign @jennybuckley @cheftako

@liggitt
Copy link
Member Author

liggitt commented Jun 1, 2018

/retest

@jennybuckley
Copy link

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 1, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jennybuckley, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@liggitt
Copy link
Member Author

liggitt commented Jun 1, 2018

/milestone v1.11
/kind bug
/priority important-soon
/status approved-for-milestone
/sig api-machinery

@k8s-ci-robot k8s-ci-robot added this to the v1.11 milestone Jun 1, 2018
@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. labels Jun 1, 2018
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process

@cheftako @jennybuckley @liggitt

Pull Request Labels
  • sig/api-machinery: Pull Request will be escalated to these SIGs if needed.
  • priority/important-soon: Escalate to the pull request owners and SIG owner; move out of milestone after several unsuccessful escalation attempts.
  • kind/bug: Fixes a bug discovered during the current release.
Help

@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit 0ea07c4 into kubernetes:master Jun 1, 2018
@liggitt liggitt deleted the debug-aggregator branch June 6, 2018 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants