-
Notifications
You must be signed in to change notification settings - Fork 18k
net/http: add clear documentation on when to drain & close a Response #60240
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
No reason for this to be a proposal, so taking it out of the proposal process. |
CC @neild |
The third sentence of the The The It's clear that there's still confusion here, but a concrete suggestion on how the documentation would be improved would help. We already state this in multiple places: The user must always close the response body, without exception. It is not necessary to drain the body first. |
for the Client.Do documentation, it states:
I think this blurb is tripping people up. I think it may be appropriate to add some
as opposed to
|
Yeah, I think that's the crux of it. The closing of Response.Body() seems to be overall properly understood, though there's apparently some idea that reading Response.Body() fully implicitly closes it? Then Client.Do() comes in which confuses things a bit, and I think a comment on
This seems to be where people are getting the idea from that as long as you don't use DefaultTransport, your transport will actually reuse HTTP/1.x "keep-alive" connections (potentially regardless of body draining and closing). |
The right thing should be to call I would not recommend going out of your way to drain response bodies unless you've got a specific and measurable performance issue that you're trying to resolve. |
If you think that changing the behavior of In practice I think it does violate the compatibility promise. It introduces other issues could dramatically impact existing programs. for example how much of a body to read - leaving this unbounded allows a server to periodically send data and keep a connection held open.
connection setup & teardown is a non-trivial cost to both client and server. it may not be an issue for small projects, but i don't think it would be uncommon for a go program to be impacted by the cost -- keep-alives are implemented for a reason. @neild do you have specific any suggestions on how to solve the problem? are you content with the current situation? |
The It might be reasonable for the client to do the same. Or perhaps not; I don't know that this would be a violation of compatibility, but it would be a user-visible change. Perhaps there should be a configuration setting for how much read-ahead we might perform. It would be useful to have some data on how much of an issue this is in practice before trying to fix it. This issue is about documentation, however. I'm not sure what we can do to improve the documentation. As I said, it's clear that people are confused, but we already document the requirement to close request bodies in many places. How can we be clearer? |
Ah, that's very interesting. I think this is why I've seen the advice previously that you don't have to drain the body on connection reuse. It's meant to only apply server side, but through all the discussions and remixes of the information that distinction may have gotten lost.
From my point of view, I think it would be more helpful to have some package level documentation on net/http that spells this out, instead of people needing to puzzle this together by looking at Response, Client.Get, Client.Do etc. It already has a comment about the fact that the client must close the response body which is one part of this issue. I think it might help to expand that section with some explicit guidance on why you may want connection draining on the client side and how to implement that. Perhaps we can include a second example, that explicitly states: "Aside from always closing the Body, if you want to enable connection reuse you also must ensure you've read the response body. In order to protect against abnormally big response sizes, it's recommended to use an io.LimitReader for this purpose". Follow up with an example that shows a It might even be feasible to update the existing example, as I don't think reading the closed body causes an issue? So (attempting) draining the connection client side is probably always safe to do? |
This is more of a documentation request / proposal to add something to the language FAQ.
There seems to currently be no authoritative answer to the following question: Do you need to drain and close an http.Response, and if so do you always need to do this or only under certain circumstances?
Throughout my own journey in Go I've seen answers varying from "no" to "only if the request is over Xkb", "always" and a lot of things in between. Doing this wrongly can lead to resource leaks so it would seem nice to at least know what the state of this is. It also seems the answer to this may have changed over time but the response tends to be "read net/http" which isn't very practical and is something one would have to redo for every Go release. The complexity of "net/http" has grown over time too and some parts of it aren't all that approachable.
I bring this up because this post is currently making the rounds: https://manishrjain.com/must-close-golang-http-response. In it, the author suggests a few things:
From the Reddit discussion, we also see a number of confused people, the advice to drain, to drain with a limit reader and that as long as you read the whole response body it is implicitly closed so you don't need to call Close().
That particular post and the ensuing confusion in the discussion including contradicting advice is only a recent example, there's a lot more of this.
It would be nice if what is expected here could be authoritatively stated in the examples & documentation of the net/http package. This seems to be a rather common issue and potential foot-gun in Go. Given Go is used extensively to write networked services, resolving ambiguity around what the client needs to do would be helpful.
The fact that this seems to confuse people to this day would also suggest http.Response might be overdue for a convenience helper that drains and/or closes the body that you can safely defer and forget about.
The text was updated successfully, but these errors were encountered: