proposal: io: add ReaderFunc and WriterFunc

[dsnet]

I propose adding the following:

type ReaderFunc func([]byte) (int, error)
func (f ReaderFunc) Read(b []byte) (int, error) {
    return f(b)
}

type WriterFunc func([]byte) (int, error)
func (f WriterFunc ) Write(b []byte) (int, error) {
    return f(b)
}

In tests, I find it common that I need an io.Reader or io.Writer with specialized behavior. Usually, this is only necessary for a single test case within a large table-driven test. In order to have a customized io.Reader, I need to declare a concrete type to implement the Read method. Since you can only declare a type with methods at the top-level, this results in the declaration potentially being located far away from the code actually using it.

The proposed ReaderFunc (and WriterFunc) allows me to create an io.Reader that 1) requires no named declarations, and 2) have the specialized Read behavior inlined with the place that actually plans on using it.

Example usage:

// Cancel the request after the first read to abruptly end the stream.
var calls int
ctx, cancel := context.WithCancel(context.Background())
req, err := http.NewRequest("POST", url, io.ReaderFunc(func(b []byte) (int, error) {
	if calls > 0 {
		cancel()
	}
	calls++
	return copy(b, []byte("whatever")), nil
}))
req = req.WithContext(ctx)

This a library-specific workaround to the several proposed language changes (see #21670, #25860, #47487, and many others). Being a non-language change, it is presumably a more palatable change than a full language change. I personally can only recall ever needing something like #21670 for io.Reader and io.Writer.

This is very similar to the existing http.HandlerFunc type.

\cc @josharian

[ianlancetaylor]

See also #47487 which tries to address this issue more generally.

[peterbourgon]

io.Reader/Writer both require something from/into which the reads/writes will occur. This is a natural semantic fit to an interface method, which must be defined on a type that would presumably serve that need. But functions are semantically stateless, so for this to make sense the function would necessarily have to close over some state, at which point you're acting as an ersatz method, anyway. (This is structurally different than e.g. http.HandlerFunc, which can fulfill its requirements statelessly.)

Minor value delivered, substantial confusion generated. -1.

[josharian]

io.Reader/Writer both require something from/into which the reads/writes will occur.

This is not true. I’ve written Readers/Writers that block forever, generate data forever, always return errors, etc.

the function would necessarily have to close over some state

Closures can be entirely local; types with methods cannot. This can help readability.

[dsnet]

Closures can be entirely local; types with methods cannot. This can help readability.

The example in the original post shows a function that closes over two variables, both on the lines immediately above it. Had that been declared as a new type, it would have been located very far from where it was actually used. That's a pretty big hit to readability.

[peterbourgon]

This is not true. I’ve written Readers/Writers that block forever, generate data forever, always return errors, etc.

That's true. io.Discard is a stateless io.Writer; rand.Rand a stateless (ish) io.Reader. My language was too severe. Still, they're exceptional cases.

Closures can be entirely local; types with methods cannot. This can help readability.

Yep, it can! I acknowledge the value this would deliver. I'm saying that value is outweighed by the costs to coherence.

[mvdan]

I'd prefer if we found a general solution like #47487 - but if that proposal is rejected, I think we should accept this one.

[renthraysk]

Would also be useful for interface hiding, for example when using io.Copy family of functions.

io.Copy(dst, ReaderFunc(f.Read))

Would be safe from an io.WriterTo implementation. And here's os having to do the same thing.

https://github.com/golang/go/blob/master/src/os/file.go#L161-L167

[jfesler]

I too prefer #47487 - but I don't think I want to for an unknown number of years for it.

In the meantime, the code outlined by @dsnet can be used in projects today, just in a different name space from "io", so this particular proposal isn't a significant blocker.

[rsc]

I will put #47487 into the regular proposal minutes and move this one to hold.

[rsc]

Placed on hold.
— rsc for the proposal review group