Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: better error detection to allow at-most-once sending for an HTTP request #44844

Open
kevinburkemeter opened this issue Mar 7, 2021 · 1 comment

Comments

@kevinburkemeter
Copy link

@kevinburkemeter kevinburkemeter commented Mar 7, 2021

Say I am making a request to a HTTP server that will send an email, charge a credit card or send a SMS to a user's phone.1 It would be bad to do those things multiple times so I would like to try once, and then stop if the request had any chance at succeeding.

If you think of an HTTP request as three steps:

  • Connect
  • Write headers and request data
  • Read data

The request can fail at any step. The safest thing to do, if you wanted to avoid sending multiple SMS messages, would be to just abort on any error.

req, err := http.Post("https://smsservice.com/...")
if err != nil {
    return err
}

There are some cases where we know it's safe to retry though. If we try to connect to a server and we get back connection refused, for example, it's safe to retry. If we get a DNS error we haven't sent any data and can retry. Generally, if it fails at the connect step, or before I have written the whole request, I can assume that it's safe to retry. Once I've written some data to the socket, I have to assume the server received it and started processing, even if I don't get any response back.

The current error handling exposed by the Go library don't do a good job of exposing this abstraction. Temporary() returns true in the case of all timeouts, but I can only retry if the timeout occurred on the connect or before the full write. That means it would be possible to send data to a misbehaving server, get Temporary() = true back, try the request again, and end up sending 2 SMS's to someone's phone.

Today, what you have to do if you want to do this right is a lot of digging into the different types of errors that can be returned into the syscall or HTTP request phase represented by the error, and then make a decision based on that.

It would be nice to have some new API designed to help make that easier. Maybe

// IsBeforeWrite returns true if an error occurred before any data was sent to the remote socket. 
// These errors indicate that the request can be safely retried, since no data was sent.
func IsBeforeWrite(err error) bool { ... }

in the golang.org/x/net/netutil package?

The advantage is, if you do this right, you can use it for any socket abstraction, for example, database reads and writes.

1 I am sure Google has great tools for this, but outside of Google, you generally accomplish these by making a HTTP request to a third party, for example Sendgrid, Stripe, or Twilio. Sometimes these services have tools to allow idempotent retry of a failed request - ie, they have internal tracking of whether a given message was sent or not, but most of the time they don't.

@kevinburkemeter kevinburkemeter changed the title net/http: at-most-once semantics for a HTTP request net/http: better error detection to allow at-most-once sending for an HTTP request Mar 7, 2021
@networkimprov
Copy link

@networkimprov networkimprov commented Mar 9, 2021

cc @ianlancetaylor re likely Proposal

@ALTree a request for new API typically goes via the proposal process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants