Skip to content

net/http: Cannot flush HTTP request headers before writing the request body #22088

@CSEMike

Description

@CSEMike

go version go1.9rc2_cl165246139 linux/amd64

It appears impossible to flush HTTP request headers for an HTTP POST before the body is written. For reasons specific to my client and server, it's important that the client be able to flush its request headers before the POST body is available.

The test below reproduces the problem.

What appears to be happening is that RoundTrip is blocked in WriteBody.
https://golang.org/src/net/http/request.go#L618

Headers have been written, but not flushed, presumably because FlushHeaders is false. I can't figure out how to induce FlushHeaders to be true, or to otherwise flush the pending request headers.
https://golang.org/src/net/http/request.go#L611

Writing anything to the request body flushes. But, a zero byte write is ignored and doesn't result in the flush I was hoping for.
https://golang.org/src/net/http/internal/chunked.go#L196

In my case, the result of this behavior is a bug. The server receiving the request times out attempting to read request headers that are delayed until the request body is available. And, the body takes ~30s to arrive since it's a hanging poll.

Test code follows.

package postflush

import (
  "io"
  "log"
  "net/http"
  "net/http/httptest"
  "net/http/httptrace"
  "runtime"
  "testing"
  "time"
)

func TestPOSTFlush(t *testing.T) {
  received := make(chan bool)
  srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    close(received)
  }))
  defer srv.Close()

  pr, pw := io.Pipe()
  req, err := http.NewRequest(http.MethodPost, srv.URL, pr)
  if err != nil {
    t.Fatalf("NewRequest: %v", err)
  }

  // Extra debugging info.
  req = req.WithContext(httptrace.WithClientTrace(req.Context(),
    &httptrace.ClientTrace{
      GetConn: func(hostport string) {
        log.Print("GetConn: ", hostport)
      },
      GotConn: func(i httptrace.GotConnInfo) {
        log.Print("reverse path: got conn: ", i)
      },
      WroteHeaders: func() {
        log.Print("reverse path: wrote headers")
      },
    }))

  go func() {
    resp, err := http.DefaultTransport.RoundTrip(req)
    if err != nil {
      t.Errorf("RoundTrip: %v", err)
      return
    }
    resp.Body.Close()
  }()

  // Writing anything unbreaks the test.
  // pw.Write([]byte("abc123"))

  select {
  case <-received:
  case <-time.After(5 * time.Second):
    t.Errorf("timed out waiting for origin to receive POST")
    buf := make([]byte, 32*1024)
    runtime.Stack(buf, true)
    t.Errorf("stacks:\n%s", string(buf))
  }
  pw.Close()
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions