Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: text/template: allow io.Reader as substitution value #29165

Closed
empijei opened this issue Dec 9, 2018 · 8 comments
Closed

proposal: text/template: allow io.Reader as substitution value #29165

empijei opened this issue Dec 9, 2018 · 8 comments

Comments

@empijei
Copy link
Contributor

@empijei empijei commented Dec 9, 2018

What version of Go are you using (go version)?

$ go version
go version go1.9.4 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/clap/go"
GORACE=""
GOROOT="/usr/lib/go-1.9"
GOTOOLDIR="/usr/lib/go-1.9/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build114129408=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

	tpl := template.Must(template.New("").Parse("{{.}}"))
	tpl.Execute(os.Stdout, strings.NewReader("gopher"))

https://play.golang.org/p/QZvPZjmezPU

What did you expect to see?

"gopher"

What did you see instead?

"{gopher 0 -1}"

Rationale

Most IO operations in go are executed on io.ReadWriters and it is very frequent to open streams, pass them around, wrap them and so on, all the way from data sources to data sinks.

One exception to this rule seems to be templates: the only way to take some text blob from a file and output it in the template is to copy the entire file in a string and then pass the string to Execute.

It would be nice to be able to use io.Readers with templates and avoid the unnecessary buffering.

A CL for the proposed change can be found here.

A previous bug mentioning this need: #25160.

@mvdan
Copy link
Member

@mvdan mvdan commented Dec 9, 2018

Sounds reasonable, given that there's no simple way to currently work around buffering all the data in memory.

I wonder how often this comes up, though. I've used templates for multiple years and I've never had the need to print an io.Reader with them. Do you have example use cases?

/cc @robpike @rogpeppe

@mvdan mvdan added the NeedsDecision label Dec 9, 2018
@empijei
Copy link
Contributor Author

@empijei empijei commented Dec 9, 2018

Sadly, none that I can link, but I've encountered this problem in several situations, mostly because I dealt a lot with net conns and external data sources that my code was just transforming (wrapping readers).

I ended up writing my own kind of templates that interleaved (*text.Template).Execute calls with io.Copy calls.

I just found the previous bug and created a small patch to support this use case for me and other people that might incur in the same issue.

Also, since most of the stdlib passes io.Readers around, it just seemed more natural and fitting to print the content of the reader instead of the reflection of its fields values, which seems like a very edge case for templates.

@ianlancetaylor ianlancetaylor added this to the Go1.13 milestone Dec 10, 2018
@mvdan mvdan changed the title text/template: add support for io.Reader Proposal: text/template: add support for io.Reader Mar 28, 2019
@gopherbot gopherbot added the Proposal label Mar 28, 2019
@mvdan mvdan modified the milestones: Go1.13, Proposal Mar 28, 2019
@mvdan
Copy link
Member

@mvdan mvdan commented Mar 28, 2019

I've re-purposed this to be a proposal, in hopes that the proposal team can make a decision.

I still think this is reasonable, though I haven't encountered the use case directly. It doesn't add any new API, and shouldn't break any existing templates. So I lean towards doing this, since none of the other template owners have spoken up.

@andybons
Copy link
Member

@andybons andybons commented Apr 17, 2019

/cc @robpike

@rsc
Copy link
Contributor

@rsc rsc commented Apr 24, 2019

Trying to do this in the template language seems quite complex. Maybe {{.}} makes sense to copy directly, but what about ``{{. | printf "%q"}}`? That would still have to buffer. There are a lot of API implications here, so it really is "new API".

Getting back to:

tpl := template.Must(template.New("").Parse("{{.}}"))
tpl.Execute(os.Stdout, strings.NewReader("gopher"))

If the template does no buffering of its own, or if we gave a way to flush the buffer, then you could define your own "copy" function that copies a reader to stdout and write {{copy .}} instead. Would that work?

@rsc rsc changed the title Proposal: text/template: add support for io.Reader proposal: text/template: allow io.Reader as substitution value Apr 24, 2019
@empijei
Copy link
Contributor Author

@empijei empijei commented Apr 25, 2019

Probably defining a copy function would solve the problem, but how would you pass the current io.Writer the template is rendering to?

@rsc
Copy link
Contributor

@rsc rsc commented Aug 13, 2019

I talked to @robpike (owner of text/template) about this a while back, and we weren't entirely sure about the implications. I'm still not. The current CL handles an io.Reader coming out of the end of execute, but what would that mean for html/template? Does html/template have to have streaming escaping converters? Should anything else change?

Does anyone want to write down the specific documentation changes that would describe the proposed change? Note that CL 25160 exempts Readers that are also Stringers or errors from being read, and I don't quite understand why.

Existing templates may also be affected unexpectedly. Perhaps a design doc is warranted here.

@empijei
Copy link
Contributor Author

@empijei empijei commented Aug 14, 2019

I ended up doing something like this, basically passing the output writer to the template and wrapping io.Copy in a helper.

The more I think about embedding this behavior in the templating language the less I'm convinced about it. I just wonder if template uses some internal buffering that might cause my workaround to break.

Closing as this can be done with current templates and adding helpers to simplify it would significantly complicate things.

Thanks for working on this.

@empijei empijei closed this Aug 14, 2019
@golang golang locked and limited conversation to collaborators Aug 13, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.