encoding/binary: Read(U)varint should return ErrUnexpectedEOF

[Stebalien]

What version of Go are you using (go version)?

$ go version
go version go1.12.7 linux/amd64

Description

binary.ReadUvarint and binary.ReadVarint currently return io.EOF when they encounter an EOF halfway to reading a varint.

This:

1. Makes it difficult to distinguish between a graceful end of the stream and the stream ending in the middle of a varint as both cases will return an EOF. Technically, one can check if the partially read varint is non-zero but that's not documented and feels like a hack.
2. Is a footgun for anyone using the standard if err != nil { return err } pattern. In this case, they'll return an EOF which usually signals "we're done", not "we read some bad data".
3. Is an incorrect usage of the EOF error.

EOF documentation:

EOF is the error returned by Read when no more input is available. Functions should return EOF only to signal a graceful end of input. If the EOF occurs unexpectedly in a structured data stream, the appropriate error is either ErrUnexpectedEOF or some other error giving more detail.

ErrUnexpectedEOF documentation:

ErrUnexpectedEOF means that EOF was encountered in the middle of reading a fixed-size block or data structure.

---

Unfortunately, as pointed out by @ianlancetaylor, fixing this would technically be an API-breaking change.

[Stebalien]

Buggy function that inspired this report: https://github.com/gogo/protobuf/blob/28a6bbf47e48e0b2220b2a244750b660c83d4942/io/varint.go#L109-L126

This function will return EOF instead of some error if the stream closes before the end of the varint.

[andybons]

@griesemer

[mxk]

Checking that a partially read varint is non-zero doesn't work if those bits are all-zero. I just ran into this problem myself and definitely think that EOF after the first byte should be converted to ErrUnexpectedEOF as done by io.ReadFull and binary.Read.

I think any existing code that relies on EOF being returned after the first byte is more likely to be broken with the author unaware of the partial-read behavior.