Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upnet/http: race in http2Transport #31192
Comments
bradfitz
changed the title
A race is detected between a bytes.Buffer generated with cmd/rpc.Pool and http2 module. An issue is raised in golang (golang/go#31192). Meanwhile, this commit disables Pool in RPC code and it generates a new 1kb of bytes.Buffer for each RPC call.
A race is detected between a bytes.Buffer generated with cmd/rpc.Pool and http2 module. An issue is raised in golang (golang/go#31192). Meanwhile, this commit disables Pool in RPC code and it generates a new 1kb of bytes.Buffer for each RPC call.
bcmills
changed the title
|
@bradfitz This might be the same issue with http. There is no guarantee that the bodyWriter is complete when roundtrip exits. |
|
Here is a standalone adapted from @vadmeste's original program but here one doesn't have to manually create certificates but also it is reduced down package main
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
"golang.org/x/net/http2"
)
func main() {
cst := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Pong")
}))
if err := http2.ConfigureServer(cst.Config, new(http2.Server)); err != nil {
log.Fatalf("Failed to configure http2 server: %v", err)
}
cst.TLS = cst.Config.TLSConfig
tr := &http.Transport{TLSClientConfig: cst.Config.TLSConfig}
if err := http2.ConfigureTransport(tr); err != nil {
log.Fatalf("Failed to configure http2 transport: %v", err)
}
tr.TLSClientConfig.InsecureSkipVerify = true
cst.StartTLS()
defer cst.Close()
// Then finally create the HTTP client with the HTTP/2 transport.
client := &http.Client{Transport: tr}
for i := 0; i < 3; i++ {
// Fill buf with data
buf := bytes.NewBuffer(bytes.Repeat([]byte("A"), 128*1024))
// Make a POST http call and read and discard response
req, err := http.NewRequest("POST", cst.URL, buf)
resp, err := client.Do(req)
if err != nil {
log.Fatalln(err)
}
if _, err := io.Copy(ioutil.Discard, resp.Body); err != nil {
log.Fatalln(err)
}
resp.Body.Close()
// Accessing buf here causes a RACE WARNING
buf.Reset()
fmt.Printf(".")
}
} |
|
@fraenkel Modifying @odeke-em 's example to use http like so: package main
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
)
func main() {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Pong")
}))
defer srv.Close()
client := srv.Client()
for i := 0; i < 100; i++ {
// Fill buf with data
buf := bytes.NewBuffer(bytes.Repeat([]byte("A"), 128*1024))
// Make a POST http call, read and discard response
req, err := http.NewRequest("POST", srv.URL, buf)
resp, err := client.Do(req)
if err != nil {
log.Fatalln(err)
}
if _, err := io.Copy(ioutil.Discard, resp.Body); err != nil {
log.Fatalln(err)
}
resp.Body.Close()
buf.Reset()
fmt.Printf(".")
}
}I do not observe a data race. The original issue is still present in gotip from
|
|
@pmalek <https://github.com/pmalek> unfortunately your modification
doesn’t create an HTTP/2 server and transport. You either want to have the
long setup version I provided, or if you are using Go1.14, use and
Unstarted httptest.Server and then enable EnableHTTP2 = true and finally
srv.StartTLS() as we do here
https://golang.org/pkg/net/http/httptest/#example_Server_hTTP2
…On Sun, Jun 14, 2020 at 4:27 AM Patryk Małek ***@***.***> wrote:
@fraenkel <https://github.com/fraenkel> Modifying @odeke-em
<https://github.com/odeke-em> 's example to use http like so:
package main
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"net/http/httptest"
)
func main() {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Pong")
}))
defer srv.Close()
client := srv.Client()
for i := 0; i < 100; i++ {
// Fill buf with data
buf := bytes.NewBuffer(bytes.Repeat([]byte("A"), 128*1024))
// Make a POST http call, read and discard response
req, err := http.NewRequest("POST", srv.URL, buf)
resp, err := client.Do(req)
if err != nil {
log.Fatalln(err)
}
if _, err := io.Copy(ioutil.Discard, resp.Body); err != nil {
log.Fatalln(err)
}
resp.Body.Close()
buf.Reset()
fmt.Printf(".")
}
}
I do not observe a data race.
The original issue is still present in gotip from go version devel
+bd486c3 Sat Jun 13 16:03:19 2020 +0000 linux/amd64 and in 1.14.2.
There is no guarantee that the bodyWriter is complete when roundtrip exits.
I'm wondering what might be the fix for this. Could we make it so that the
bodyWriter *is* complete when roundtrip exits?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#31192 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABFL3V6DCJY3VPZ6VVTLIRDRWSX3VANCNFSM4HC2FYMA>
.
|
|
@odeke-em My bad. But the TLS is also required to trigger this bug so both |
|
It took me a bit to reproduce the issue and then find the statement that covers this situation. https://golang.org/pkg/net/http/#RoundTripper The body isn't closed until net/http.(*http2clientStream).writeRequestBody() returns. |
That's interesting because as far as I can see there's no way of telling outside of the e.g. |
|
Looking at this further, the issue is when a GOAWAY is sent from the server all outstanding streams are sent a response errClientConnGotGoAway. However, cs.writeRequestBody() may still be executing. |
|
Good news, its easy to fix. Just have to create a test that always fails. |
|
Change https://golang.org/cl/253259 mentions this issue: |
|
Could you please add this fix to next milestone, as Kubernetes affected. |
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env)?go envOutputWhat did you do?
What did you expect to see?
No race detector warning
What did you see instead?