Skip to content

net/http: first byte of outgoing request body is sometimes lost #17071

Closed
@jacobsa

Description

@jacobsa
% go version
go version devel +8086e7c Mon Sep 12 06:12:35 2016 +0000 darwin/amd64

(Apologies for not having been able to reproduce this hermetically. I suspect maybe it has something to do with SSL, which I don't know how to set up locally. I'm pretty sure the bug does lie in the Go client though; please keep reading.)

Here is a program that makes a bunch of calls to the Google Cloud Storage API. The calls are all identical. Although this is a heavily-reduced test case and they are not valid according to the GCS API (in particular they're missing credentials), they all do supply valid JSON in the request body.

Despite this, every time I run the program, after at most a few tens of seconds I receive an HTTP 400 parse error from GCS:

% go run http2_bug.go
[...]
Req body:
{"bucket":"foo","name":"bar"}
[...]
Resp body:
{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "parseError",
    "message": "Parse Error"
   }
  ],
  "code": 400,
  "message": "Parse Error"
 }
}

Status code: 400

I happen to have insider knowledge that this charmingly terse error message from GCS indicates that the JSON in the request body could not be parsed. (If you want details, please contact me internally within Google.) In particular, the server on the other end does not receive the opening { character.

Go's own HTTP2 debugging agrees.

% GODEBUG=http2debug=2 go run http2_bug.go

With this invocation, among lots of other output I see

2016/09/12 20:46:54 http2: Framer 0xc420242180: wrote HEADERS flags=END_HEADERS stream=455 len=7
2016/09/12 20:46:54 http2: Framer 0xc420242180: wrote DATA stream=455 len=1 data="\""
2016/09/12 20:46:54 http2: Framer 0xc420242180: wrote DATA stream=455 len=27 data="bucket\":\"foo\",\"name\":\"bar\"}"
2016/09/12 20:46:54 http2: Framer 0xc420242180: wrote DATA flags=END_STREAM stream=455 len=0 data=""
2016/09/12 20:46:54 http2: Framer 0xc420242180: read HEADERS flags=END_HEADERS stream=455 len=94
2016/09/12 20:46:54 http2: Framer 0xc420242180: read DATA flags=END_STREAM stream=455 len=171 data="{\n \"error\": {\n  \"errors\": [\n   {\n    \"domain\": \"global\",\n    \

The http2 package has forgotten to write the opening curly brace. In contrast, everything works fine if I use GODEBUG=http2client=0 to downgrade to HTTP1.

FYI, I have observed this from Sydney (Google office and at home, multiple computers) and from a computer within a Google data center in the US. I've seen this in Go 1.7.1 and Go 1.6.1, as well as at HEAD.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions