net/http: better error detection to allow at-most-once sending for an HTTP request

[kevinburke1]

Say I am making a request to a HTTP server that will send an email, charge a credit card or send a SMS to a user's phone.¹ It would be bad to do those things multiple times so I would like to try once, and then stop if the request had any chance at succeeding.

If you think of an HTTP request as three steps:

• Connect
• Write headers and request data
• Read data

The request can fail at any step. The safest thing to do, if you wanted to avoid sending multiple SMS messages, would be to just abort on any error.

req, err := http.Post("https://smsservice.com/...")
if err != nil {
    return err
}

There are some cases where we know it's safe to retry though. If we try to connect to a server and we get back connection refused, for example, it's safe to retry. If we get a DNS error we haven't sent any data and can retry. Generally, if it fails at the connect step, or before I have written the whole request, I can assume that it's safe to retry. Once I've written some data to the socket, I have to assume the server received it and started processing, even if I don't get any response back.

The current error handling exposed by the Go library don't do a good job of exposing this abstraction. Temporary() returns true in the case of all timeouts, but I can only retry if the timeout occurred on the connect or before the full write. That means it would be possible to send data to a misbehaving server, get Temporary() = true back, try the request again, and end up sending 2 SMS's to someone's phone.

Today, what you have to do if you want to do this right is a lot of digging into the different types of errors that can be returned into the syscall or HTTP request phase represented by the error, and then make a decision based on that.

It would be nice to have some new API designed to help make that easier. Maybe

// IsBeforeWrite returns true if an error occurred before any data was sent to the remote socket. 
// These errors indicate that the request can be safely retried, since no data was sent.
func IsBeforeWrite(err error) bool { ... }

in the golang.org/x/net/netutil package?

The advantage is, if you do this right, you can use it for any socket abstraction, for example, database reads and writes.

¹ I am sure Google has great tools for this, but outside of Google, you generally accomplish these by making a HTTP request to a third party, for example Sendgrid, Stripe, or Twilio. Sometimes these services have tools to allow idempotent retry of a failed request - ie, they have internal tracking of whether a given message was sent or not, but most of the time they don't.

[networkimprov]

cc @ianlancetaylor re likely Proposal

@ALTree a request for new API typically goes via the proposal process.