Refine DirectService

[carllerche]

This issue proposes to change the contract of the poll_service function on DirectService

Background

The current implementation of DirectService is as follows:

pub trait DirectService<Request> {
    type Response;
    type Error;

    type Future: Future<Item = Self::Response, Error = Self::Error>;

    fn poll_ready(&mut self) -> Poll<(), Self::Error>;

    fn poll_service(&mut self) -> Poll<(), Self::Error>;

    fn poll_close(&mut self) -> Poll<(), Self::Error>;

    fn call(&mut self, req: Request) -> Self::Future;
}

This mirrors the Service trait while including two additional functions: poll_service and poll_close.

Currently, poll_service is documented as:

Returns Ready whenever there is no more work to be done until call
is invoked again.

Note that this method may return NotReady even if there are no
outstanding requests, if the service has to perform non-request-driven
operations (e.g., heartbeats).

Proposal

Most services, including HTTP 1.0 services, need a way to signal graceful close. poll_service should be updated such that Ready is returned when the service has gracefully closed without the user requesting a close by calling poll_close.

Implementations of DirectService should make a best effort to return Ready from poll_service when the service is closed. However, there is no guarantee that poll_service will return Ready before a call to poll_ready returning an error due to the service being closed. It is sometimes impossible to detect the service being closed without attempting to check for readiness.

This also would allow for a class of middleware that are able to "ignore" inner service errors by mapping service errors to a graceful close. This would be useful for middleware like ConnectionPool, which needs to be able to tell the difference between a fatal inner service error and an inner service gracefully going away.

Refs #136

cc/ @jonhoo @seanmonstar