New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wait for post-hooks to run before checking the healthz output #73516

Merged
merged 1 commit into from Feb 2, 2019

Conversation

@mfojtik
Copy link
Contributor

mfojtik commented Jan 30, 2019

What type of PR is this?
/kind failing-test

What this PR does / why we need it:

Under some circumstances (when starting go routines is slow?) this test can fail as the result will return [-]poststarthook/delegate-post-start-hook failed (reason withheld). This is because we start checking the output before the hook is ran.

Sample failure:

expected [+]ping ok
        [+]log ok
        [-]wrapping-health failed: reason withheld
        [-]delegate-health failed: reason withheld
        [+]poststarthook/generic-apiserver-start-informers ok
        [+]poststarthook/delegate-post-start-hook ok
        [+]poststarthook/wrapping-post-start-hook ok
        healthz check failed
        , got [+]ping ok
        [+]log ok
        [-]wrapping-health failed: reason withheld
        [-]delegate-health failed: reason withheld
        [+]poststarthook/generic-apiserver-start-informers ok
        [-]poststarthook/delegate-post-start-hook failed: reason withheld
        [+]poststarthook/wrapping-post-start-hook ok
        healthz check failed

With this change we will wait until the hooks ran and only after that check the healthz endpoint result.

NONE
@sttts

This comment has been minimized.

Copy link
Contributor

sttts commented Jan 30, 2019

/lgtm
/approve

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jan 30, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mfojtik, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@soltysh
Copy link
Contributor

soltysh left a comment

/lgtm
/retest

@roycaihw

This comment has been minimized.

Copy link
Member

roycaihw commented Jan 31, 2019

/assign @logicalhan

@soltysh

This comment has been minimized.

Copy link
Contributor

soltysh commented Feb 1, 2019

/retest

@soltysh

This comment has been minimized.

Copy link
Contributor

soltysh commented Feb 1, 2019

Not sure why the bot didin't pick up the approval from @sttts, adding the label manually.

@soltysh soltysh added the approved label Feb 1, 2019

@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented Feb 1, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented Feb 1, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Feb 1, 2019

@mfojtik: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-kops-aws 8a2f709 link /test pull-kubernetes-e2e-kops-aws

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-ci-robot k8s-ci-robot merged commit bc40649 into kubernetes:master Feb 2, 2019

17 checks passed

cla/linuxfoundation mfojtik authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-kops-aws Context retired without replacement.
pull-kubernetes-godeps Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment