io: add an `Err` field to `LimitedReader`

[DeedleFake]

New Proposal

As per @ianlancetaylor's suggestion, this proposal is now to add an Err field to io.LimitedReader. If not nil, this field's value will be returned when trying to read past the limit imposed by the reader instead of the default io.EOF.

Original Proposal

For various reasons, I had a need to use MaxBytesReader() in a situation where a ResponseWriter wasn't easily available, so I looked through the code to see if it was safe to pass nil and found that all it does is check a type assertion and then call an unexported method on it:

type requestTooLarger interface {
	requestTooLarge()
}
if res, ok := l.w.(requestTooLarger); ok {
	res.requestTooLarge()
}

While it's true that because of this it's safe to pass it a nil ResponseWriter, it feels quite odd to rely on undocumentated behavior of this kind.

Proposal

Several ways to clean this situation up come to mind:

1. Add an alternative in the http package that doesn't need a ResponseWriter. Because of how it works, it could actually just return the same implementation, but the function will be more obvious in its usage.
2. Document how the implementation uses the ResponseWriter and explicitly state that a nil ResponseWriter is valid.
3. Add a reimplementation of MaxBytesReader() in io that doesn't use a ResponseWriter at all and isn't tied to http behavior, but includes the other differences. This could actually be done, for the most part, by just adding an Err error field to io.LimitedReader that, if non-nil, is the error returned when the limit is reached instead of io.EOF.

In whichever case, the documentation for MaxBytesReader() should probably be filled in a bit. It is quite odd that something called Reader requires a type that is primarily an io.Writer implementation for non-obvious reasons and doesn't mention it anywhere in the documentation.

[jimmyfrasche]

I have forked io.LimitedReader to return a custom error at least three times.

[ianlancetaylor]

If the proposed function doesn't use http.ResponseWriter, then I can't see any reason that it should live in the net/http package. That seems to reduce this to

type MyLimitedReader struct {
    r   io.LimitedReader
    err error
}

func (mlr *MyLimitedReader) Read(p []byte) (int, error) {
    n, err := mlr.Read(p)
    if err == io.EOF {
        err = mlr.err
    }
    return n, er
}

Is that correct?

If so, perhaps this proposal should be rewritten to add an Err field io.LimitedReader.

[earthboundkid]

I like the new proposal. Returning io.EOF makes io.LimitedReader pretty unhelpful. But I think it would also be good to document that http.MaxBytesReader accepts nil ResponseWriter.

[bcmills]

I have forked io.LimitedReader to return a custom error at least three times.

FWIW, my moreio.LimitedWriter has an Err field too:

[earthboundkid]

Can someone tag this proposal as under review?

[xeoncross]

As an alternative to defining your own error, I just created a package and export the error so the caller can check against it.

// Throws an error if the reader is bigger than limit.
var ErrSizeExceeded = errors.New("stream size exceeded")

type MaxBytesReader struct {
	io.ReadCloser       // reader object
	N             int64 // max bytes remaining.
}

func NewMaxBytesReader(r io.ReadCloser, limit int64) *MaxBytesReader {
	return &MaxBytesReader{r, limit}
}

func (b *MaxBytesReader) Read(p []byte) (n int, err error) {
	if b.N <= 0 {
		return 0, ErrSizeExceeded
	}

	if int64(len(p)) > b.N {
		p = p[0:b.N]
	}

	n, err = b.ReadCloser.Read(p)
	b.N -= int64(n)
	return
}

[earthboundkid]

That would break current users of io.LimitReader/LimitedReader.

I misunderstood the proposal.

[ianlancetaylor]

@carlmjohnson It is already so tagged. We'll get to it soon, I think.

[gopherbot]

Change https://go.dev/cl/396215 mentions this issue: io: add an Err field to LimitedReader

[rsc]

This seems reasonable. Thanks for the proposal.