net/http, net/http/httputil: Chunked reader doesn't read the final two bytes of the chunked data

[gopherbot]

by jgrahamc:

What is the expected output?

It's expected that the function beginChunk() will read the trailing CRLF from the final
chunk size when doing chunked encoding. It does not. The function is as follows:

func (cr *chunkedReader) beginChunk() {
    // chunk-size CRLF
    var line string
    line, cr.err = readLine(cr.r)
    if cr.err != nil {
        return
    }
    cr.n, cr.err = strconv.ParseUint(line, 16, 64)
    if cr.err != nil {
        return
    }
    if cr.n == 0 {
        cr.err = io.EOF
    }
}

When cr.n == 0 the readLine() has consumed 0\r\n but there's an additional \r\n at the
end (see RFC2616 section 3.6).

What do you see instead?

There are two bytes left in the buffer when done.

Which compiler are you using (5g, 6g, 8g, gccgo)?

6g

Which operating system are you using?

64 bit Linux

Which version are you using?  (run 'go version')

go version go1.0.3

Please provide any additional information below.

[gopherbot]

Comment 1 by jgrahamc:

From the RFC:
     Chunked-Body   = *chunk
                        last-chunk
                        trailer
                        CRLF
       chunk          = chunk-size [ chunk-extension ] CRLF
                        chunk-data CRLF
       chunk-size     = 1*HEX
       last-chunk     = 1*("0") [ chunk-extension ] CRLF

[bradfitz]

Comment 3:

The last-chunk is a zero and a CRLF, both of which we consume.
The final CRLF you're referring to is part of the Chunked-Body (after the trailer, which
we also read).  We're not a Chunked-Body reader, because an io.Reader doesn't know how
to read HTTP trailers.  The chunked reader only reads until the EOF, and then lets the
HTTP ReadResponse read the trailer and final CRLF.
If anything, this is a documentation fix.
At least, that's my recollection, but you looked at this more recently.  Does that
explanation make sense?

Status changed to WaitingForReply.

[gopherbot]

Comment 4 by jgrahamc:

I've looked at the code and your explanation is correct. My original report was wrong.
The readTrailer code while read the CRLF as you describe.  This can be closed.

[davecheney]

Comment 5:

Status changed to Retracted.