proposal: text/template: allow io.Reader as substitution value

[empijei]

What version of Go are you using (go version)?

$ go version
go version go1.9.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/clap/go"
GORACE=""
GOROOT="/usr/lib/go-1.9"
GOTOOLDIR="/usr/lib/go-1.9/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build114129408=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

	tpl := template.Must(template.New("").Parse("{{.}}"))
	tpl.Execute(os.Stdout, strings.NewReader("gopher"))

https://play.golang.org/p/QZvPZjmezPU

What did you expect to see?

"gopher"

What did you see instead?

"{gopher 0 -1}"

Rationale

Most IO operations in go are executed on io.ReadWriters and it is very frequent to open streams, pass them around, wrap them and so on, all the way from data sources to data sinks.

One exception to this rule seems to be templates: the only way to take some text blob from a file and output it in the template is to copy the entire file in a string and then pass the string to Execute.

It would be nice to be able to use io.Readers with templates and avoid the unnecessary buffering.

A CL for the proposed change can be found here.

A previous bug mentioning this need: #25160.

[mvdan]

Sounds reasonable, given that there's no simple way to currently work around buffering all the data in memory.

I wonder how often this comes up, though. I've used templates for multiple years and I've never had the need to print an io.Reader with them. Do you have example use cases?

/cc @robpike @rogpeppe

[empijei]

Sadly, none that I can link, but I've encountered this problem in several situations, mostly because I dealt a lot with net conns and external data sources that my code was just transforming (wrapping readers).

I ended up writing my own kind of templates that interleaved (*text.Template).Execute calls with io.Copy calls.

I just found the previous bug and created a small patch to support this use case for me and other people that might incur in the same issue.

Also, since most of the stdlib passes io.Readers around, it just seemed more natural and fitting to print the content of the reader instead of the reflection of its fields values, which seems like a very edge case for templates.

[mvdan]

I've re-purposed this to be a proposal, in hopes that the proposal team can make a decision.

I still think this is reasonable, though I haven't encountered the use case directly. It doesn't add any new API, and shouldn't break any existing templates. So I lean towards doing this, since none of the other template owners have spoken up.

[andybons]

/cc @robpike

[rsc]

Trying to do this in the template language seems quite complex. Maybe {{.}} makes sense to copy directly, but what about ``{{. | printf "%q"}}`? That would still have to buffer. There are a lot of API implications here, so it really is "new API".

Getting back to:

tpl := template.Must(template.New("").Parse("{{.}}"))
tpl.Execute(os.Stdout, strings.NewReader("gopher"))

If the template does no buffering of its own, or if we gave a way to flush the buffer, then you could define your own "copy" function that copies a reader to stdout and write {{copy .}} instead. Would that work?

[empijei]

Probably defining a copy function would solve the problem, but how would you pass the current io.Writer the template is rendering to?

[rsc]

I talked to @robpike (owner of text/template) about this a while back, and we weren't entirely sure about the implications. I'm still not. The current CL handles an io.Reader coming out of the end of execute, but what would that mean for html/template? Does html/template have to have streaming escaping converters? Should anything else change?

Does anyone want to write down the specific documentation changes that would describe the proposed change? Note that CL 25160 exempts Readers that are also Stringers or errors from being read, and I don't quite understand why.

Existing templates may also be affected unexpectedly. Perhaps a design doc is warranted here.