net/http: Client support for Expect: 100-continue #3665

bradfitz · May 23, 2012

The http Client should probably support Expect: 100-continue.  The server does.

It does introduce a new round-trip, though, so we should probably only send it when the
data size exceeds some threshold.  Probably the same threshold we use for consuming
request bodies when a handler doesn't (maxPostHandlerReadBytes).

I think we could see if we know the size of the request body (just as NewRequest does,
checking for a few known types), but if the Body is non-nil and the ContentLength is
<=0 (unknown), we instead slurp up the Body into a bytes.Buffer to see if it's >=
someThreshold.  If so, we replace the Body with a io.MultiReader(slurped, rest) and
proceed to do a 100-continue.  If not, we blast it away HTTP/1.0-style.

Comment 1:

See also issue #2184, different but related.

bradfitz · Aug 7, 2012

Comment 1:

See also issue #2184, different but related.

Comment 2:

Labels changed: added priority-later, removed priority-triage.

rsc · Sep 12, 2012

Comment 2:

Labels changed: added priority-later, removed priority-triage.

Comment 3:

Labels changed: added go1.1.

rsc · Sep 12, 2012

Comment 3:

Labels changed: added go1.1.

Comment 4:

Labels changed: added go1.1maybe, removed go1.1.

rsc · Dec 9, 2012

Comment 4:

Labels changed: added go1.1maybe, removed go1.1.

Comment 5:

Adding the "Suggested" label, if somebody wants to take this on.  I expect it to be
somewhat involved, though.
It should probably be opt-in:  users will need to add "Expect": "100-continue" to their
request before sending it.
And then in transport.go, when we're going to write the request, write and flush the
headers, create a channel, and then wait on the channel response and a 1 second timer
(like curl).  If after 1 second no response, send the body anyway.  If the channel gives
you a non-100 HTTP response, use that.  If the channel gives a 100, start writing the
body.

Labels changed: added suggested, size-l.

bradfitz · Dec 14, 2012

Comment 5:

Adding the "Suggested" label, if somebody wants to take this on.  I expect it to be
somewhat involved, though.
It should probably be opt-in:  users will need to add "Expect": "100-continue" to their
request before sending it.
And then in transport.go, when we're going to write the request, write and flush the
headers, create a channel, and then wait on the channel response and a 1 second timer
(like curl).  If after 1 second no response, send the body anyway.  If the channel gives
you a non-100 HTTP response, use that.  If the channel gives a 100, start writing the
body.

Labels changed: added suggested, size-l.

Comment 6:

Owner changed to ---.

bradfitz · Feb 20, 2013

Comment 6:

Owner changed to ---.

Comment 7:

[The time for maybe has passed.]

Labels changed: removed go1.1maybe.

rsc · Mar 12, 2013

Comment 7:

[The time for maybe has passed.]

Labels changed: removed go1.1maybe.

Comment 8:

Sent out https://golang.org/cl/8166045

bradfitz · Mar 30, 2013

Comment 8:

Sent out https://golang.org/cl/8166045

Comment 9:

This issue was updated by revision a79df7b.

R=golang-dev, dsymonds, dave, jgrahamc
CC=golang-dev
https://golang.org/cl/8166045

bradfitz · Mar 30, 2013

Comment 9:

This issue was updated by revision a79df7b.

R=golang-dev, dsymonds, dave, jgrahamc
CC=golang-dev
https://golang.org/cl/8166045

Comment 10:

I think this fixes issue #2184.

andybalholm · Mar 30, 2013

Comment 10:

I think this fixes issue #2184.

Comment 11:

Labels changed: added go1.2maybe.

rsc · Jul 30, 2013

Comment 11:

Labels changed: added go1.2maybe.

Comment 12:

Labels changed: added feature.

rsc · Jul 30, 2013

Comment 12:

Labels changed: added feature.

Comment 13:

Won't happen for 1.2.

Labels changed: removed go1.2maybe.

robpike · Aug 29, 2013

Comment 13:

Won't happen for 1.2.

Labels changed: removed go1.2maybe.

Comment 14:

Just a quick note to let you know that node. Is only send a 100 if the request sends an
expect 100. That seems like a good idea.
Also, does the http client supports expect 100 with timeout

mattetti · Oct 13, 2013

Comment 14:

Just a quick note to let you know that node. Is only send a 100 if the request sends an
expect 100. That seems like a good idea.
Also, does the http client supports expect 100 with timeout

Comment 15:

Labels changed: added go1.3maybe.

rsc · Nov 27, 2013

Comment 15:

Labels changed: added go1.3maybe.

Comment 16:

Labels changed: removed feature.

rsc · Nov 27, 2013

Comment 16:

Labels changed: removed feature.

Comment 17:

Labels changed: added release-none, removed go1.3maybe.

rsc · Dec 4, 2013

Comment 17:

Labels changed: added release-none, removed go1.3maybe.

Comment 18:

Labels changed: added repo-main.

rsc · Dec 4, 2013

Comment 18:

Labels changed: added repo-main.

Comment 19:

I was going create a patch for this, but noticed that the concurrent read/write loops
handle this surprisingly well as is. A very easy improvement would be to flush headers
immediately if expectsContinue during the initial write.
One question regarding brafitz's earlier comment, do we actually *need* to block sending
the body (seems ambiguous in the spec). I'd actually prefer the current behavior with
the added flush, as it would get a head start on sending data, and it looks like the
client still shuts down immediately upon a non 100-continue response.

jbardin · Mar 31, 2014

Comment 19:

I was going create a patch for this, but noticed that the concurrent read/write loops
handle this surprisingly well as is. A very easy improvement would be to flush headers
immediately if expectsContinue during the initial write.
One question regarding brafitz's earlier comment, do we actually *need* to block sending
the body (seems ambiguous in the spec). I'd actually prefer the current behavior with
the added flush, as it would get a head start on sending data, and it looks like the
client still shuts down immediately upon a non 100-continue response.

I believe this is the cause for issues I'm having PUT-ing to a server that redirects me. The body in the initial PUT request is prematurely consumed, despite my having set Expect: 100-continue. If there is Expect: 100-continue in the request, the body needs to be withheld from the request until a 100 Continue is received from the server.

anacrolix · May 10, 2015

I believe this is the cause for issues I'm having PUT-ing to a server that redirects me. The body in the initial PUT request is prematurely consumed, despite my having set Expect: 100-continue. If there is Expect: 100-continue in the request, the body needs to be withheld from the request until a 100 Continue is received from the server.

Although I'm not sure if it were a right implementation, I've sent my small patch for this issue.
https://go-review.googlesource.com/#/c/10091/

I hit this premature consuming issue on my project. So I would appreciate if someone review this CL.

matope · May 14, 2015

Although I'm not sure if it were a right implementation, I've sent my small patch for this issue.
https://go-review.googlesource.com/#/c/10091/

I hit this premature consuming issue on my project. So I would appreciate if someone review this CL.

CL https://golang.org/cl/10091 mentions this issue.

gopherbot · May 14, 2015

CL https://golang.org/cl/10091 mentions this issue.

@bradfitz ping

mattn · May 29, 2015

@bradfitz ping

The Go tree is currently in freeze for the Go 1.5 release. The freeze started on May 1st, and this was sent after May 1st, and this was not tagged as a Go 1.5-blocking issue.

As such, this has to wait until Go 1.6.

I've marked that CL with R=close so it doesn't show up on our dashboard, but pinging this thread at any time will make it reappear. Please reply to this bug after August 1st and we can begin the code review.

Thanks for working on this, though!

bradfitz · May 29, 2015

The Go tree is currently in freeze for the Go 1.5 release. The freeze started on May 1st, and this was sent after May 1st, and this was not tagged as a Go 1.5-blocking issue.

As such, this has to wait until Go 1.6.

I've marked that CL with R=close so it doesn't show up on our dashboard, but pinging this thread at any time will make it reappear. Please reply to this bug after August 1st and we can begin the code review.

Thanks for working on this, though!

@bradfiz OK, I catch on your timeline. I'll ping you again this August!

@mattn Thanks for your pinging!

matope · May 29, 2015

@bradfiz OK, I catch on your timeline. I'll ping you again this August!

@mattn Thanks for your pinging!

I've been having issues with this as welI (using Go with CouchDB: http://stackoverflow.com/questions/30541591/large-put-requests-from-go-to-couchdb ) and I really hope this gets in sometime so I can remove the ugly hack-around in my code 👍

rhinoman · Jul 31, 2015

I've been having issues with this as welI (using Go with CouchDB: http://stackoverflow.com/questions/30541591/large-put-requests-from-go-to-couchdb ) and I really hope this gets in sometime so I can remove the ugly hack-around in my code 👍

For golang-backed servers, is there a way to increase the max size before it starts the 100-continue protocol?

drnic · Aug 24, 2015

For golang-backed servers, is there a way to increase the max size before it starts the 100-continue protocol?

My body seems to be only 2000+ characters of JSON, so am not sure what triggers to 100-continue sequence (and causes my golang client to hang until timeout)

drnic · Aug 24, 2015

My body seems to be only 2000+ characters of JSON, so am not sure what triggers to 100-continue sequence (and causes my golang client to hang until timeout)

@bradfitz can we get this in now?
On 25/08/2015 4:24 AM, "Dr Nic Williams" notifications@github.com wrote:

My body seems to be only 2000+ characters of JSON, so am not sure what
triggers to 100-continue sequence (and causes my golang client to hang
until timeout)

—
Reply to this email directly or view it on GitHub
#3665 (comment).

anacrolix · Aug 25, 2015

@bradfitz can we get this in now?
On 25/08/2015 4:24 AM, "Dr Nic Williams" notifications@github.com wrote:

My body seems to be only 2000+ characters of JSON, so am not sure what
triggers to 100-continue sequence (and causes my golang client to hang
until timeout)

—
Reply to this email directly or view it on GitHub
#3665 (comment).

I replied with some initial review feedback.

bradfitz · Aug 25, 2015

I replied with some initial review feedback.

@bradfitz Thank you for your feedback! I revised my patch a few weeks ago.
I think my patch got better but I'm not sure if it is enough.
If you are free, could you please look through that?

matope · Sep 24, 2015

@bradfitz Thank you for your feedback! I revised my patch a few weeks ago.
I think my patch got better but I'm not sure if it is enough.
If you are free, could you please look through that?

Replied on Gerrit.

bradfitz · Sep 24, 2015

Replied on Gerrit.

@bradfitz Thank you for your review comments.

I've just updated my CL on Gerrit.
I fixed a issue that my previous code couldn't flush request header before waiting response.
I added some explicit codes for canceling sending body when the response status was not 100.

matope · Oct 3, 2015

@bradfitz Thank you for your review comments.

I've just updated my CL on Gerrit.
I fixed a issue that my previous code couldn't flush request header before waiting response.
I added some explicit codes for canceling sending body when the response status was not 100.

bradfitz added priority-later Suggested labels Mar 31, 2014

rsc added this to the Unplanned milestone Apr 10, 2015

rsc removed the priority-later label Apr 10, 2015

rsc referenced this issue Apr 10, 2015
Closed
net/http: something about 100-continue #9735

rsc removed release-none labels Apr 14, 2015

anacrolix referenced this issue May 10, 2015
Closed
net/http: Redirect conditions are incorrect, support 307/308 #10767

bradfitz modified the milestones: Go1.6, Unplanned May 29, 2015

harshavardhana referenced this issue Jun 24, 2015
Closed
http: Verify if we need to get http client support for 100 Continue #101

emicklei referenced this issue Sep 20, 2015
Closed
remove curl #2

bradfitz closed this in dab143c Oct 17, 2015

sielaq referenced this issue Feb 16, 2016
Open
The 100-Continue mechanism doesn't work correctly #54

golang locked and limited conversation to collaborators Oct 17, 2016

gopherbot added the FrozenDueToAge label Oct 17, 2016

golang/go

Join GitHub today

net/http: Client support for Expect: 100-continue #3665

Comments

bradfitz commented May 23, 2012

This comment has been minimized.

bradfitz commented Aug 7, 2012

This comment has been minimized.

rsc commented Sep 12, 2012

This comment has been minimized.

rsc commented Sep 12, 2012

This comment has been minimized.

rsc commented Dec 9, 2012

This comment has been minimized.

bradfitz commented Dec 14, 2012

This comment has been minimized.

bradfitz commented Feb 20, 2013

This comment has been minimized.

rsc commented Mar 12, 2013

This comment has been minimized.

bradfitz commented Mar 30, 2013

This comment has been minimized.

bradfitz commented Mar 30, 2013

This comment has been minimized.

andybalholm commented Mar 30, 2013

This comment has been minimized.

rsc commented Jul 30, 2013

This comment has been minimized.

rsc commented Jul 30, 2013

This comment has been minimized.

robpike commented Aug 29, 2013

This comment has been minimized.

mattetti commented Oct 13, 2013

This comment has been minimized.

rsc commented Nov 27, 2013

This comment has been minimized.

rsc commented Nov 27, 2013

This comment has been minimized.

rsc commented Dec 4, 2013

This comment has been minimized.

rsc commented Dec 4, 2013

This comment has been minimized.

jbardin commented Mar 31, 2014

bradfitz added priority-later Suggested labels Mar 31, 2014

rsc added this to the Unplanned milestone Apr 10, 2015

rsc removed the priority-later label Apr 10, 2015

rsc referenced this issue Apr 10, 2015

rsc removed release-none labels Apr 14, 2015

This comment has been minimized.

anacrolix commented May 10, 2015

anacrolix referenced this issue May 10, 2015

This comment has been minimized.

matope commented May 14, 2015

This comment has been minimized.

gopherbot commented May 14, 2015

This comment has been minimized.

mattn commented May 29, 2015

This comment has been minimized.

bradfitz commented May 29, 2015

bradfitz modified the milestones: Go1.6, Unplanned May 29, 2015

This comment has been minimized.

matope commented May 29, 2015

harshavardhana referenced this issue Jun 24, 2015

This comment has been minimized.

rhinoman commented Jul 31, 2015

This comment has been minimized.

drnic commented Aug 24, 2015

This comment has been minimized.

drnic commented Aug 24, 2015

This comment has been minimized.

anacrolix commented Aug 25, 2015

This comment has been minimized.

bradfitz commented Aug 25, 2015

emicklei referenced this issue Sep 20, 2015

This comment has been minimized.

matope commented Sep 24, 2015

This comment has been minimized.

bradfitz commented Sep 24, 2015

This comment has been minimized.

matope commented Oct 3, 2015

bradfitz closed this in `dab143c` Oct 17, 2015