Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return grpc serving status in health check errors #8726

Merged
merged 2 commits into from
Sep 25, 2020

Conversation

amenzhinsky
Copy link
Contributor

Hi there!

This PR adds serving status to gRPC health check errors, because sometimes it's not clear what causes a failure, SERVICE_UNKNOWN due to misconfiguration or NOT_SERVING when a service is shutting down.

@hashicorp-cla
Copy link

hashicorp-cla commented Sep 22, 2020

CLA assistant check
All committers have signed the CLA.

Copy link
Contributor

@pierresouchay pierresouchay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about improving the message with non-int values, but human-readable one?

if response == nil || response.Status != hv1.HealthCheckResponse_SERVING {
return ErrGRPCUnhealthy
if response.Status != hv1.HealthCheckResponse_SERVING {
return fmt.Errorf("gRPC %s serving status: %s", target, response.Status)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

response.Status is an int, might be improved:

statusTxt, ok := hv1.HealthCheckResponse_ServingStatus_name[response.Status]
if !ok {
   statusTxt = fmt.Sprintf("UnexpectedGRPCStatus:=%d", response.Status)
}
return fmt.Errorf("gRPC Error: %s for target %s", statusTxt, target)

Copy link
Contributor Author

@amenzhinsky amenzhinsky Sep 25, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

response.Status already has String method, that's why I put %s there.

func (m *HealthCheckResponse) String() string { return proto.CompactTextString(m) }

e.g. grpc_health_v1.HealthCheckResponse_NOT_SERVING.String() produces NOT_SERVING

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, sorry :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think UnexpectedGRPCStatus can confuse people with general gRPC status codes.

Copy link
Contributor

@pierresouchay pierresouchay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dnephin dnephin added theme/health-checks Health Check functionality type/enhancement Proposed improvement or new feature labels Sep 25, 2020
Copy link
Contributor

@dnephin dnephin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR! LGTM

I added a changelog entry for this change

@dnephin dnephin merged commit 6200325 into hashicorp:master Sep 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme/health-checks Health Check functionality type/enhancement Proposed improvement or new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants