net/http:

[libjs]

What version of Go are you using (go version)?

$ go version
go version go1.21.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.1'
GCCGO='/usr/bin/gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2527636589=/tmp/go-build -gno-record-gcc-switches'

What did you do?

server.go code

package main

import (
   "fmt"
   "net/http"
)

type handler struct {
}

func (h *handler) ServeHTTP (w http.ResponseWriter, r *http.Request) {
   go func() {
      ctx := r.Context()
      <- ctx.Done()
      log(ctx.Err())
   }()
   log(r.Method + " " + r.RequestURI + " " + r.Proto)
   d := w.Header()
   d.Set("Server", "foo/1.0")
   if origin := r.Header.Get("Origin"); "" != origin {
      d.Set("Access-Control-Allow-Origin", origin)
   }
   switch r.Method {
   case "PUT":
      w.WriteHeader(http.StatusRequestEntityTooLarge)
   case "OPTIONS":
      d.Set("Access-Control-Allow-Methods", "OPTIONS, GET, HEAD, PUT, DELETE")
      d.Set("Access-Control-Allow-Headers", "Content-Type")
      d.Set("Access-Control-Allow-Max-Age", "86400")
      d.Set("Access-Control-Allow-Credentials", "true")
      w.WriteHeader(http.StatusNoContent)
   }
   log("Done")
}

func log (i any) {
   fmt.Printf("%v\n", i)
}

func main () {
   srv := &http.Server{
      Addr: ":8888",
      Handler: &handler{},
   }
   log(srv.ListenAndServe())
}

I do (file size 10MB)

$ go run server.go
$ wget -t1 -qSO - --method=PUT --body-file=10MB.mp4 'http://localhost:8888/'

server log

PUT / HTTP/1.1
Done
context canceled

clent log empty and connection reset

$ echo $?
4

man wget (4 = Network failure.)

---

ok, I do (file size 1MB)

$ wget -t1 -qSO - --method=PUT --body-file=1MB.mp3 'http://localhost:8888/'

client log now

  HTTP/1.1 413 Request Entity Too Large
  Server: foo/1.0
  Date: Sat, 09 Sep 2023 07:44:06 GMT
  Content-Length: 0

What did you expect to see?

expected behavior, client receives response

What did you see instead?

client does not receive a response, and connection is closed

---

Additionally, when using encryption and HTTP 2, the error does not recur; it seems that the context is canceled due to a timeout and the client does not have time to read the response when the header Content-Length contains a sufficiently large value. And, if the request is made from the browser, the browser sends another repeat request

[seankhliao]

you're not reading the body

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

[libjs]

I am reporting an BUG, this is the most simplified version, I have no right to provide you with the source code of the project. behavior does not depend on reading the body. so that the answer is that the body is too large, should the entire body of size 100500 GB be read? in addition, when using HTTP 2 this bug is not observed

---

just repeat this test and everything will become clear if the text does not describe it clearly

[libjs]

https://datatracker.ietf.org/doc/html/rfc9110#name-413-content-too-large

15.5.14. 413 Content Too Large
The 413 (Content Too Large) status code indicates that the server is refusing to process a request because the request content is larger than the server is willing or able to process. The server MAY terminate the request, if the protocol version in use allows it; otherwise, the server MAY close the connection. If the condition is temporary, the server SHOULD generate a Retry-After header field to indicate that it is temporary and after what time the client MAY try again.

and with this bug there is no way to transfer any information to the client, he believes that the error is network and can repeat requests endlessly

[andig]

@libjs your handler still needs to be correct, that is handle the body data.

[libjs]

@libjs your handler still needs to be correct, that is handle the body data.

no, it shouldn't process the body if it's too big, especially if it's known from the header Content-Length for example

---

and this is not my handler, this is a test that you can easily repeat by adding anything, the result will be the same, I spent a week on this bug

[libjs]

I repeat once again, to save resources, if after receiving the headers or in the process of reading the body, there is a need to respond to the client, then this is impossible without finishing reading the body, which can be very large

---

and everything would be solved by using HTTP 2, but proxies use HTTP 1.1, it is this fact that breaks the ability to respond to the client

[libjs]

in fact, this bug is a vulnerability, if you force the developer to read the entire body to be able to correctly complete the request, then it is easy to carry out a DDOS attack on such a server, sending it a body with the maximum possible body size, and this takes up at least network resources

[seankhliao]

your client is closing the connection before the response can be written because you've filled up whatever buffers there may be. there's nothing the server can do.

[libjs]

your client is closing the connection before the response can be written because you've filled up whatever buffers there may be. there's nothing the server can do.

It’s not my client that does this, it’s the server that closes the connection and this happens to any client, use any browser
I have my own implementation of the HTTP protocol for Golang, and it has been used in production for more than 5 years, working with 500k+ persistent connections, I personally know how to solve this problem without using the native HTTP module, I’m just checking whether Golang developers can solve this problem

[libjs]

Just answer, what do you need to understand the problem? I'll provide everything

[seankhliao]

see https://pkg.go.dev/net/http#ResponseController.EnableFullDuplex

[libjs]

see https://pkg.go.dev/net/http#ResponseController.EnableFullDuplex

tnx, im test it. and let you know the result

[libjs]

see https://pkg.go.dev/net/http#ResponseController.EnableFullDuplex

sorry, its not working

package main

import (
   "fmt"
   "net/http"
)

// wget -t1 -qSO - --method=PUT --body-file=video.mp4 'http://localhost:8888/'

type handler struct {
}

func (h *handler) ServeHTTP (w http.ResponseWriter, r *http.Request) {
   rc := http.NewResponseController(w)
   log(rc.EnableFullDuplex())
   go func() {
      ctx := r.Context()
      <- ctx.Done()
      log(ctx.Err())
   }()
   log(r.Method + " " + r.RequestURI + " " + r.Proto)
   d := w.Header()
   d.Set("Server", "foo/1.0")
   if origin := r.Header.Get("Origin"); "" != origin {
      d.Set("Access-Control-Allow-Origin", origin)
   }
   switch r.Method {
   case "PUT":
      w.WriteHeader(http.StatusRequestEntityTooLarge)
   case "OPTIONS":
      d.Set("Access-Control-Allow-Methods", "OPTIONS, GET, HEAD, PUT, DELETE")
      d.Set("Access-Control-Allow-Headers", "Content-Type")
      d.Set("Access-Control-Allow-Max-Age", "86400")
      d.Set("Access-Control-Allow-Credentials", "true")
      w.WriteHeader(http.StatusNoContent)
   }
   log("Done")
}

func log (i any) {
   fmt.Printf("%v\n", i)
}

func main () {
   srv := &http.Server{
      Addr: ":8888",
      Handler: &handler{},
   }
   log(srv.ListenAndServe())
}

server log

nil
PUT / HTTP/1.1
Done
context canceled

and client dont receive response