Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: client action when doing redirection #30689

Closed
mmpei opened this issue Mar 8, 2019 · 6 comments

Comments

Projects
None yet
3 participants
@mmpei
Copy link

commented Mar 8, 2019

go version 1.11.2.
i use http.client to get an object from an object storage server continually.
the server will response with status code 307 and a redirection url.
my code like:

for ; true;  {
		go func() {
			url := "http://storage.server.com/key/object1"
			client := http.Client{
			}
			resp, err := client.Get(url)
			if err != nil {
				return 
			}
			if resp.Body != nil {
				defer resp.Body.Close()
			}
			//	ioutil.ReadAll(resp.Body)    // this special line
		}()
	}

if uncomment the special line it works as my expection, object will be downloaded continually and data stream could be watched from network card,

# dstat -n 1
 recv  send
1679M 4241k
1728M 4632k
1710M 3984k
1699M 4007k
1777M 3957k
 804M 2591k
1967M 8078k
1558M 4155k
1308M 2821k
1484M 3211k
1430M 3076k
1610M 3915k

if comment the special line, the http.client will also send a request to the redirection url, but no as much as data stream be watched in network card. i doubt that the objects didn't be downloaded.

# dstat -n 1
  10M 2289k
  11M 2289k
  12M 1797k
  13M 2428k
 210k  149k
9356k 2103k
  12M 2361k
  11M 2073k
  11M 2040k

my question is what's the difference between processing http response with and without 'ioutil.ReadAll(resp.Body)'.
could anyone help me to resolve my confusion? thanks.

@andybons andybons changed the title http client action when doing redirection net/http: client action when doing redirection Mar 8, 2019

@andybons andybons modified the milestones: Unreleased, Unplanned Mar 8, 2019

@andybons

This comment has been minimized.

Copy link
Member

commented Mar 8, 2019

@mmpei

This comment has been minimized.

Copy link
Author

commented Mar 12, 2019

@bradfitz would you help to answer? thank you.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

Sorry, but I don't understand the question or problem.

@mmpei

This comment has been minimized.

Copy link
Author

commented Mar 12, 2019

sorry, i update my question. @bradfitz
in brief my question is what's the difference between processing http response with and without 'ioutil.ReadAll(resp.Body)'.
i find that no data stream in network card if without 'ioutil.ReadAll(resp.Body)' and doubt whether the object be downloaded.

by the way there is a redirection in http invoke.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 12, 2019

If you close a response body without reading from it, the net/http package has the right to just close the TCP connection, as that's usually more efficient that reading bytes just to discard them. See docs at https://golang.org/pkg/net/http/#Client.Do

@bradfitz bradfitz closed this Mar 12, 2019

@mmpei

This comment has been minimized.

Copy link
Author

commented Mar 12, 2019

so if the returned error is nil, does http.client guarantee that all the requests is sent correctly and all response data is fetched to local even if the reponse data is large?
you can see my test result above, it seems that no data stream in network card when comment 'ioutil.ReadAll()'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.