net/http: document Transport errors more, types of errors returned, how to check canceled, etc

[erikdubbelboer]

At the moment the only wat to know if an http.Request was canceled is to check the error string. If http.errRequestCanceled is exported this becomes much easier.

I can submit a change to gerrit if needed?

[davecheney]

@erikdubbelboer what would you do if errRequestCancalled was exported ? If you want to retry the request, would it be better if the error returned implemented a Temporary() bool method ?

[erikdubbelboer]

@davecheney we use http for some internal communication. It's around 5000 requests per second with a 100ms timeout before we cancel the request. At the moment we log all errors but would like to ignore errors that aren't important to us. If errRequestCancalled was exported we could type assert this error and ignore it in a more robust way than doing a strings.HasSuffix and hoping the messages never changes.

I'm not sure which errors you would call temporary.

For example we already ignore connection refused errors using:

    if uerr, ok := err.(*url.Error); ok {
        if noerr, ok := uerr.Err.(*net.OpError); ok {
            if scerr, ok := noerr.Err.(*os.SyscallError); ok {
                if scerr.Err == syscall.ECONNREFUSED {
                    ignore = true
                }
            }
        }
    }

[tombergan]

Don't you already know the request was canceled since your code canceled the request, or does cancellation happen across an API boundary? Also, I think multiple different errors can be returned depending on when the request is cancelled (currently errRequestCanceled or errRequestCanceledConn).

[erikdubbelboer]

Yes it's across an API boundary. Both sides are internal code so we could fix this internally but I thought it would also be nice for other people to be able to detect this error this way.

Shouldn't in theory all error types the Go API returns be exported so a user can always detect them reliably without resorting to string comparisons?

[tombergan]

Shouldn't in theory all error types the Go API returns be exported so a user can always detect them reliably without resorting to string comparisons?

I don't entirely disagree. I have some ugly code that detects net/http's errTimeout by inspecting the error string. It would be nice if that error was exported.

The counter-argument is that this forces the API to have a mostly-stable list of possible errors, which can be difficult. How do you decide which errors to export? If your errors are too abstract, the error messages are less useful. If your errors are too low-level, there can be heavy churn as the code changes and old errors are deprecated (e.g., see issue #15150).

Strictly speaking, I don't think all errors should be exported, but maybe there's a small set of stable errors to export? ErrRequestCanceled and ErrResponseHeadersTimedOut are both good candidates since they relate directly to behaviors controlled by the net/http API.

[bradfitz]

@nightlyone might have suggestions for what to do here, per his #15192

[nightlyone]

@tombergan having a type satisfying https://godoc.org/github.com/nightlyone/errorclass#Canceled for errRequestCanceled might help this effort.

So the user of the http package will know what actions to take from these error classifications.

It seems the errTimeout is already correctly classified and Temporary() will will return true there. It will even report that it is a timeout.

The exact error is usually not needed to decide what to do once the application is running, since complete error enumerations is an endless, never complete task.

But what is useful, is to support the decision, whether I should retry (Temporary() == true), ignore it (in the cancel case) or report it.

[tombergan]

@nightlyone

The exact error is usually not needed to decide what to do once the application is running, since complete error enumerations is an endless, never complete task. But what is useful, is to support the decision, whether I should retry (Temporary() == true), ignore it (in the cancel case) or report it.

If only it were that simple! Before seeing this issue, you might have reasonably concluded that Temporary() was "good enough" to make programmatic decisions regarding HTTP errors. Then this bug added another case, which is to ignore canceled errors. Your errorclass seems to handle this case.

Then a future bug wants to know if the request timed out because Transport.ResponseHeaderTimeout was exceeded, or because Transport.TLSHandshakeTimeout was exceeded, or because the req.Context exceeded its deadline, or because of a DNS timeout, or because of a TCP connect timeout, etc. You cannot answer these questions with a simple IsTimeout() method on the error.

For example: I maintain a large proxy service that uses net/http. Rather than logging net/http errors as strings, we convert net/http errors into a custom enum error space (mostly because it's easy to verify that an enum doesn't contain PII, compared to an error string, which can contain arbitrary info; also, enums are easier to process than strings at large scales). Part of that service is ~70 lines of code that tries to convert the error returned by RoundTrip() or Body.Read() into an error in our custom enum space. This code actually isn't too bad. The standard library already exports types like net.OpError, net.DNSError, and os.SyscallError. However, we check for ResponseHeaderTimeout errors by checking for error strings that contain "timeout awaiting response headers". This introspection would be unnecessary if net/http's errTimeout was exported.

I agree with your comment that "complete error enumerations is an endless, never complete task", but that comment is also overly pessimistic. Here's a simple rule: if an API exports N knobs, then it should also export N error types or vars, where each error means that the operation failed due to the corresponding knob. For net/http.RoundTrip, I think three errors could be exported following this rule:

errTimeout               // Transport.ResponseHeaderTimeout
errRequestCanceled       // Transport.CancelRequest, Request.Cancel, Request.Context
tlsHandshakeTimeoutError // Transport.TLSHandshakeTimeout

Note that errRequestCanceledConn doesn't need to be exported since it's controlled by the same knob as errRequestCanceled. Plus, the caller can use the new httptrace API to figure out when the error happened.

(Footnote: I think you could make a case that RoundTrip should return Request.Context().Err() instead of errRequestCanceled when Request.Context() completes, to preserve that original error.)

[bradfitz]

(Footnote: I think you could make a case that RoundTrip should return Request.Context().Err() instead of errRequestCanceled when Request.Context() completes, to preserve that original error.)

@ianlancetaylor recently submitted 0b5f2f0 which does that.

Maybe that's enough for this bug?

[tombergan]

Yep, I think that would satisfy @erikdubbelboer's specific request.

WDYT about also exporting errTimeout and tlsHandshakeTimeoutError because of this suggested rule:

if an API exports N knobs, then it should also export N error types or vars, where each error means that the operation failed due to the corresponding knob.

[bradfitz]

Both of those already have a IsTimeout() bool method which return true.

I don't want the error package to be a minefield of error types and variables obscuring the useful parts.

I also don't want the implementation to be restricted to returning specific error variables over time, constraining the ability to refactor things. I'd rather promise properties of the errors, like that its IsTimeout() == true (ideally with a helper somewhere to get at it) or things like "if the context becomes done, you get back its error, wrapped in a URLError", which is kinda implicit already from the http docs.

Actually, it's not. The only mention of url.Error at https://golang.org/pkg/net/http/ is this one appearance:

        // If CheckRedirect returns an error, the Client's Get
        // method returns both the previous Response (with its Body
        // closed) and CheckRedirect's error (wrapped in a url.Error)
        // instead of issuing the Request req.

But nothing else talks about *url.Error.

Before we start ad-hoc fixing stuff like exporting single variables, I'd rather have a concerted effort to document the status quo where one already exists, make inconsistent things consistent, then document them, and write new tests locking in behavior for the documentation. Maybe we have some of those tests already, and some (but not enough) docs.

But I don't think the answer is more error variables right now.

Witness this error variable:

        // Deprecated: ErrWriteAfterFlush is no longer used.
        ErrWriteAfterFlush = errors.New("unused")

It's in the public docs and was only documented to be no longer used after it was discovered that it was vestigial (#15150). We don't want more of those.

Do you want to specifically know when a TLS handshake timeout happens? Can you not already tell between httptrace hooks and the Timeout() bool method?

[tombergan]

Can you not already tell between httptrace hooks

I think that's right -- these questions can be answered via the httptrace package. So I think you can close this bug.

I also don't want the implementation to be restricted to returning specific error variables over time, constraining the ability to refactor things.

For the record (not important; you can close this bug), I don't think this is a problem for the rule proposed above because an error is exported only if a knob exists in the public API. You're constrained to support that error for as long as you're constrained to support that API knob.

[bradfitz]

@tombergan, I'll just slightly repurpose this bug instead, changing its title.

[glasser]

As described in #18272, it would also be nice if you could easily tell if an error returned by http.Transport.RoundTrip came from communicating with the target server or from reading request.Body (or a third place I'm not considering).

[bradfitz]

I've started some notes in https://docs.google.com/a/golang.org/document/d/15J1ypbcK7OJ2IJ6TEgB3TitNPTCpo0kAwBbrZu2tzPo/edit?usp=sharing about what Transport actually does today.

Hopefully people don't depend on the current behavior too much. I seem to recall other bugs though where people have made deep assumptions about specific error types returned by RoundTrip, even though nothing is promised.

[gopherbot]

CL https://golang.org/cl/34381 mentions this issue.

[bradfitz]

Well, I looked into this, but it's too much to document for Go 1.8, not that I think we'd even want to document it in its current state. (See earlier comment)

[gopherbot]

CL https://golang.org/cl/37495 mentions this issue.

[bradfitz]

Duplicate of #9424. (Not sure why I never noticed earlier)