x/net/http2: Transport is leaking streams on broken Body

[euroelessar]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

linux, amd64

What did you do?

1. Create golang.org/x/net/http2.Transport
2. Send POST request with non-nil Body
3. Body returns an error on Read call

What did you expect to see?

Client fails the request and resets the stream.

What did you see instead?

Client fails the request and leaks the stream (on both client and server sides).

[odeke-em]

Thank you filing this issue @euroelessar and welcome to Go!

So to help with a prognosis, proper fix for this issue and for posterity, it would be helpful to paste in here a runnable repro because the conditions that you are described are part of the usual x/net/http2.Transport usages.

In the meantime I'll page some http2 folks @rs @tombergan @bradfitz

[euroelessar]

Thanks, @odeke-em.
Pull request has a test which exposes the issue. Though let me try to combine some standalone example as well.

[euroelessar]

Standalone example: https://gist.github.com/euroelessar/6f4535db11b8bf024d85253cdaa1db1f
It fails against master x/net/http2 package (deadline on the last GET call due to exceeding concurrency limit).

[rs]

In case of request body write error, the stream is not reset/forgotten. The following patch fixes the issue, though I'm not sure a cancel is the right response to such problem spec-wise:

diff --git a/http2/transport.go b/http2/transport.go
index 9d1f2fa..b5c8701 100644
--- a/http2/transport.go
+++ b/http2/transport.go
@@ -1060,7 +1060,12 @@ func (cc *ClientConn) roundTrip(req *http.Request) (res *http.Response, gotErrAf
                        default:
                        }
                        if err != nil {
-                          return nil, cs.getStartedWrite(), err
+                               startedWrite := cs.getStartedWrite()
+                               if startedWrite {
+                                       cc.writeStreamReset(cs.ID, ErrCodeCancel, nil)
+                               }
+                               cc.forgetStreamID(cs.ID)
+                               return nil, startedWrite, err
                        }
                        bodyWritten = true
                        if d := cc.responseHeaderTimeout(); d != 0 {

[gopherbot]

Change https://golang.org/cl/132715 mentions this issue: http2: don't leak streams on broken body

[enocom]

This issue seems to be the same one behind googleapis/google-cloud-go#1211 and hashicorp/vault#5419.

I see this is attached to the Go 1.12 milestone, but the CL hasn't received any feedback. Are there plans to look at this?

[bradfitz]

I plan to look at it. Still targeted towards Go 1.12. (but could also be backported to Go 1.11.x)

[bradfitz]

Approved & submitted into x/net. Needs bundle into std.

[enocom]

Thanks for the fast response!

[lbernail]

Thank you!
It would be great if the fix could be backported to 1.11

[bradfitz]

@gopherbot, please backport to Go 1.11.

[gopherbot]

Backport issue(s) opened: #28673 (for 1.11).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[gopherbot]

Change https://golang.org/cl/152080 mentions this issue: net/http: update bundled x/net/http2