health check package

[yutongp]

refer to this: https://github.com/uber-go/fx/blob/master/modules/uhttp/health.go#L32

I like the idea behind our internal gocommon health package. When we used it, we made some twists.

1. instead of a Check func(), we defined an interface:

// Checkable is anything implemented Health method.
type Checkable interface {
	// Health takes a context and return error if health check failed.
	Health(ctx context.Context) error
}
// also func
type CheckableFunc func(ctx context.Context) error
func (f CheckableFunc) Health(ctx context.Context) error { return f(ctx) }

so all gateway or storage interfaces can be health checked once they implemented this interface. We can even automatically wrap the dependency's meta.thrift to it.

2. we also add a Checker struct can do:

{
    // Add adds a Checkable map to a given name to Checker as a hard dependency.
    Add(name string, check Checkable)
    
    // AddSoft adds a Checkable map to a given name to Checker as a soft dependency.
    // If the check on soft dependency fails, checker will generate a warning message without make health check fail.
    AddSoft(name string, check Checkable)

    // ThriftHealthFunc returns a thrift health check function.
    ThriftHealthFunc() thrift.HealthFunc

    // HTTPHealthFunc returns a http health check function.
    HTTPHealthFunc(ctx context.Context) http.HandlerFunc
}

I think this is quite flexible compare to one global health checker since nowadays a service can serving more than one "service" with both soft and hard dependencies.
Also http.HandlerFunc can be mount under any route and thrift.HealthFunc can be mounted to tchannel.Service served on this server with Server.RegisterHealthHandler easily.
One issue is YARPC doesn't have Server.RegisterHealthHandler right now. I opened a ticket to track this: yarpc/yarpc-go#733.

3. The remaining are the same. host name, timestamp, pid, response time, status will all be returned in health report.

What's you guys' thought on these interface/struct? What other features or integrations you think should be included in health check package?

[ascandella]

Yes, we have plans for a much more advanced version where components can expose introspection data in a plugin model that you can then inspect from outside the service. It's on our internal issue tracker. Let's chat tomorrow about how to avoid overlap between GH issues and our internal roadmap.