Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

text/template: no nice way to escape/suppress newlines #9969

Closed
eloff opened this issue Feb 23, 2015 · 43 comments
Closed

text/template: no nice way to escape/suppress newlines #9969

eloff opened this issue Feb 23, 2015 · 43 comments
Assignees
Milestone

Comments

@eloff
Copy link

@eloff eloff commented Feb 23, 2015

In many text/template scenarios, when creating text files (e.g. csv) for machine consumption, or text documents for human consumption, it's desirable to control the newlines output by the template. Currently this involves jumping through hoops by moving template tags around, making the template very unreadable. A simple e.g \ character at the end of a line would be nice to escape the newline so it doesn't end up in the output text, but the template tags can still be organized nicely in the template itself.

Seems there was already a request for this on go-nuts: https://groups.google.com/forum/#!topic/golang-nuts/H_d6P6av8nk

@adg
Copy link
Contributor

@adg adg commented Feb 23, 2015

However this might work, it would need to be backward-compatible with existing templates. That rules out \ at the end of a line.

@eloff
Copy link
Author

@eloff eloff commented Feb 24, 2015

Good point. How about nunjucks system of adding a - at the start and/or end of a block to eat leading/trailing whitespace? Nunjucks does that: http://mozilla.github.io/nunjucks/templating.html#whitespace-control

{{range .foos -}} // eats trailing whitespace
{{.}}
{{-end}} // eats leading whitespace (\n from previous line)
@adg
Copy link
Contributor

@adg adg commented Feb 24, 2015

Seems like a reasonable approach. Can you give me an example of where this might help, so that I can judge the feature in context?

@eloff
Copy link
Author

@eloff eloff commented Feb 24, 2015

Sure, the thread had an example of generating a tsv file I think, but my
use case is generating program code. I end up with a lot of empty lines
where I had control tags, range tags, if tags, etc.

so I have something that looks like:

{{with .BitShift}}
    {{if ne . 0}}val >>= {{ . }}{{end}}
{{end}}

{{with $mask := .BitMask}}
{{if ne $mask 0 }}val &= 0x{{ printf "%x" $mask }}{{end}}
{{if ne $mask 0 | and $.IsSigned }}
val = val<<63 | val>>1
{{end}}
{{end}}

Which produces a lot of empty lines. I have to put non-related things on
the same line, make very long lines, etc to get the formatting I want. With
some whitespace eating feature I could do:

{{with .BitShift-}} // eats the newline and the indentation on the next

line
{{if ne . 0}}val >>= {{ . }}{{end}}
{{end-}}
{{with $mask := .BitMask-}}
{{if ne $mask 0 }}val &= 0x{{ printf "%x" $mask }}{{end}}
{{if ne $mask 0 | and $.IsSigned -}}
val = val<<63 | val>>1
{{end-}}
{{end-}}

Not sure if the dash character is the best choice, but something like that
to suppress newlines and indentation before or after tags would do the
trick.

On Mon, Feb 23, 2015 at 10:12 PM, Andrew Gerrand notifications@github.com
wrote:

Seems like a reasonable approach. Can you give me an example of where this
might help, so that I can judge the feature in context?


Reply to this email directly or view it on GitHub
#9969 (comment).

@josharian
Copy link
Contributor

@josharian josharian commented Feb 24, 2015

I've also frequently run into this while generating code and been frustrated by it. go/format can take care of multiple newlines, but it never goes from one newline to zero.

@mikioh mikioh changed the title text/template no nice way to escape/suppress newlines text/template: no nice way to escape/suppress newlines Feb 24, 2015
@adg
Copy link
Contributor

@adg adg commented Feb 25, 2015

The current way to avoid a newline is to do this:

Some text without a {{/*
*/}}newline in the middle.

which is a little verbose for such a minor signal. It would be helpful to have some way to signal that we don't want a newline.

  1. Do we want the signal to be part of existing actions? (As in, {{end FOO}} where FOO is some token that doesn't collide with the current syntax.) Or, should "no new line" be an action of its own? ({{FOO}} for example) The former has the benefit of brevity, the latter has the benefit of being unambiguous and is probably cleaner to implement
  2. Regardless of the answer to question 1, what should FOO be? A dash (-) is one idea, but there are many other things we can use. Back slash (\) seems more natural to me.
Some text without a {{\}}
newline in the middle.

A list of items:
{{range .Items \}}
{{.}} {{end}}

Should produce (where Items is [1,2,3,4]):

Some text without a newline in the middle.

A list of items:
1 2 3 4 

Hey @robpike what do you think?

@minux
Copy link
Member

@minux minux commented Feb 25, 2015

@eloff
Copy link
Author

@eloff eloff commented Feb 25, 2015

Or it can be a method on template, e.g.
template.New(...).WithLineEscapes().Parse(...)

On Tue, Feb 24, 2015 at 7:32 PM, Minux Ma notifications@github.com wrote:

Could we add some kind of pre-template configuration options?
{{ option +line-continuation }}
or something like that.

and then we can just use the most natural syntax for line continuation:
line 1
line 2

It also preserves compatibility with existing templates.

we can also make the line continuation character (or character sequence)
configurable.


Reply to this email directly or view it on GitHub
#9969 (comment).

@minux
Copy link
Member

@minux minux commented Feb 25, 2015

@dworld
Copy link

@dworld dworld commented Mar 7, 2015

I expect it as a builtin function, for example.

now:

        {{ range .Route }}
            {{ if contains .Method "GET" }} server.Get("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}})
            {{end}} {{ if contains .Method "POST" }} server.Post("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}}) {{end}}  {{ end }}

use a builtin functin EAT_NEXT_NEWLINE:

        {{ range .Route }}
            {{ if contains .Method "GET" }} server.Get("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}}) {{ EAT_NEXT_NEWLINE }} {{ end }} {{ end }}
            {{ if contains .Method "POST" }} server.Get("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}}) {{ end }} {{ EAT_NEXT_NEWLINE }} 
        {{ end }}
@cespare
Copy link
Contributor

@cespare cespare commented Mar 7, 2015

I run into this almost every time I use */template. I just did today: I was generating big SQL prepared statements using text/template and I had to choose between making the template easy-to-read and making the generated statement easy-to-read.

@asa511
Copy link

@asa511 asa511 commented Mar 10, 2015

I am using consul-template to generate text config files using a Go template. The templates can get quite complicated with nested ranges and conditionals. Making output result readable leads to ugly and unreadable code. Needs to be fixed at the earliest.

@loa
Copy link

@loa loa commented Mar 12, 2015

+1

@zeeraw
Copy link

@zeeraw zeeraw commented Mar 18, 2015

I'm also generating configuration which at some point will need to be read by humans. I really like the idea of {{- end }} for leading whitespace and {{ end -}} for trailing. +1

@ydnar
Copy link

@ydnar ydnar commented Apr 4, 2015

We’ve used strings.Replace with a continuation char:

var (
    src = `{{range .Items}} \
        Hello, \
        {{.Name}} \
    {{end}}
    `
    t = template.Must(template.New("go").Parse(strings.Replace(src, "\\\n", "", -1)))
)

https://github.com/zonedb/zonedb/blob/master/build/generate.go#L85

@rsc rsc added this to the Unplanned milestone Apr 10, 2015
@joncalhoun
Copy link

@joncalhoun joncalhoun commented Apr 26, 2015

I agree that something like this would be useful. My biggest complaint with templates thus far is that they are hard to make human-readable without sacrificing the quality of the output.

That said, I suspect that there may be a lot of unique cases. Eg in my case I only wanted to trim newlines, so I used the following:

leading := regexp.MustCompile("(\n)*[{]{2}[-][ ]*")
tempBytes = leading.ReplaceAll(tempBytes, []byte("{{"))
trailing := regexp.MustCompile("[ ]*[-][}]{2}(\n)*")
tempBytes = trailing.ReplaceAll(tempBytes, []byte("}}"))

If you do address this, it would be nice to have a little flexibility but I don't really know how far you can go without it snowballing into too much.

@kaelumania
Copy link

@kaelumania kaelumania commented May 5, 2015

+1

@gyuho
Copy link
Contributor

@gyuho gyuho commented May 14, 2015

+1, this might help us write more readable code:

Let's say we want to create this text:

CREATE TABLE IF NOT EXISTS my.table  (
    key VARCHAR(100) PRIMARY KEY NOT NULL,
    value1 INTEGER,
    value2 INTEGER
);

I had to:

package main

import (
    "bytes"
    "fmt"
    "html/template"
    "log"
)

func main() {
    queryStruct := struct {
        SchemaName string
        TableName  string
        Slice      []map[string]string
        LastIndex  int
    }{
        "my",
        "table",
        []map[string]string{
            map[string]string{"key": "VARCHAR(100) PRIMARY KEY NOT NULL"},
            map[string]string{"value1": "INTEGER"},
            map[string]string{"value2": "INTEGER"},
        },
        2,
    }
    tb := new(bytes.Buffer)
    if err := template.Must(template.New("tmpl").Parse(queryTmpl)).Execute(tb, queryStruct); err != nil {
        log.Fatal(err)
    }
    fmt.Println(tb.String())
}

var queryTmpl = `CREATE TABLE IF NOT EXISTS {{.SchemaName}}.{{.TableName}}  ({{$lastIndex := .LastIndex}}
{{range $index, $valueMap := .Slice}}{{if ne $lastIndex $index}}{{range $key, $value := $valueMap}} {{$key}} {{$value}},{{end}}
{{else}}{{range $key, $value := $valueMap}} {{$key}} {{$value}}{{end}}
{{end}}{{end}});`

http://play.golang.org/p/gl5CJWVry7

Thanks,

@progrium
Copy link

@progrium progrium commented May 14, 2015

Just as a reference, Python Jinja's whitespace control is done with an optional global configuration to trim whitespace from lines with only a template block, as well as fine grained control based on the - pattern mentioned earlier.

http://jinja.pocoo.org/docs/dev/templates/#whitespace-control

The - pattern is also present in Ruby's ERB and others. If I were to prefer a syntax, it would be the one used everywhere else.

@mgcrea
Copy link

@mgcrea mgcrea commented May 20, 2015

👍 for {{- end }}-like syntax, to me this is really a must have.

@progrium
Copy link

@progrium progrium commented May 21, 2015

{{+ would be rad to include the previous line whitespace for every line output of the block. so pipeline expression output doesn't have to care about indentation when indentation is important for the outer file (like yaml)

@josharian
Copy link
Contributor

@josharian josharian commented May 27, 2015

@robpike any thoughts about this? I just bumped into it again. I may be a princess, but this is definitely a pea.

@steve-jansen
Copy link

@steve-jansen steve-jansen commented May 28, 2015

+1 for {{-end}} syntax, already familiar with this from Ruby/ERB

@azihsoyn
Copy link

@azihsoyn azihsoyn commented Jun 1, 2015

👍 +1 for {{-end}}

@mickep76
Copy link

@mickep76 mickep76 commented Jun 5, 2015

Lines that only has code statements should probably not generate a newline at all. It is nice to keep them on a separate line for readability.

Input: [1,2,3]

{{range .}}
Apple: {{.}}
{{end}}

Result:

Apple: 1

Apple: 2

Apple: 3

As for the other part {{-end}} seems similar to what Jinja does. So why not.

@jasonwbarnett
Copy link
Contributor

@jasonwbarnett jasonwbarnett commented Jun 13, 2015

+1. Both Ruby ERB and Jinja do somewhat similar things and I think it's an absolute requirement to have something similar in Go. Otherwise templating output looks stupid. Almost as stupid as this comment I'm leaving.

@dverbeek84
Copy link

@dverbeek84 dverbeek84 commented Jun 19, 2015

+1 for ERB / Jinja style

@obieq
Copy link

@obieq obieq commented Jul 18, 2015

+1 for some sort of resolution, preferably {{-end}}

@chasebolt
Copy link

@chasebolt chasebolt commented Aug 6, 2015

+1 for {{- end }} syntax, matches well with ERB templates

@andrewwebber
Copy link

@andrewwebber andrewwebber commented Aug 14, 2015

+1 for {{- end }}

@n-rodriguez
Copy link

@n-rodriguez n-rodriguez commented Aug 16, 2015

+1 for ERB / Jinja style

@rsc rsc modified the milestones: Go1.6Early, Unplanned Aug 26, 2015
@JakeWarner
Copy link

@JakeWarner JakeWarner commented Sep 4, 2015

+1 for the suggested "{{- end}}" as well.

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 8, 2015

CL https://golang.org/cl/14391 mentions this issue.

robpike added a commit that referenced this issue Sep 9, 2015
…en actions

Borrowing a suggestion from the issue listed below, we modify the lexer to
trim spaces at the beginning (end) of a block of text if the action immediately
before (after) is marked with a minus sign. To avoid parsing/lexing ambiguity,
we require an ASCII space between the minus sign and the rest of the action.
Thus:

	{{23 -}}
	<
	{{- 45}}

produces the output
	23<45

All the work is done in the lexer. The modification is invisible to the parser
or any outside package (except I guess for noticing some gaps in the input
if one tracks error positions). Thus it slips in without worry in text/template
and html/template both.

Fixes long-requested issue #9969.

Change-Id: I3774be650bfa6370cb993d0899aa669c211de7b2
Reviewed-on: https://go-review.googlesource.com/14391
Reviewed-by: Andrew Gerrand <adg@golang.org>
@robpike robpike closed this Sep 10, 2015
@mickep76
Copy link

@mickep76 mickep76 commented Sep 11, 2015

Thanks!

1 similar comment
@loa
Copy link

@loa loa commented Sep 11, 2015

Thanks!

@zeeraw
Copy link

@zeeraw zeeraw commented Sep 11, 2015

Wohoo, thanks!

@cgcgbcbc
Copy link

@cgcgbcbc cgcgbcbc commented Sep 11, 2015

when will next release come?

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 11, 2015

The next release is scheduled for February 1, 2016.

@brandoncole
Copy link

@brandoncole brandoncole commented Sep 11, 2015

This will be fantastic to have!

sr added a commit to sr/grpcinstrument that referenced this issue Dec 19, 2015
The only difference are the extra newlines but that will have to wait
until after Go 1.6 is out:

golang/go#9969
@mopfeffe
Copy link

@mopfeffe mopfeffe commented Jan 27, 2016

This comment is too late..but I've only recently joined the word of go templating.
The discussion above is a good one...Perhaps
Would another possible solution be to surpress blank lines by default if there is no output from
the template instructions on that line?

Example:

Text
Tex2
{{range $key, $pairs := ...some tree}}
Text3 $key

{{end}}

The lines containing {{range.}} and {{end}} produce no actual output - so supress the blank line.

@davecheney
Copy link
Contributor

@davecheney davecheney commented Jan 27, 2016

This has been implemented, please see the go 1.6 release notes.

@thomasleveil
Copy link

@thomasleveil thomasleveil commented Jan 14, 2017

and the link to the Go 1.6 release notes

First, it is now possible to trim spaces around template actions, which can make template definitions more readable. A minus sign at the beginning of an action says to trim space before the action, and a minus sign at the end of an action says to trim space after the action. For example, the template

{{23 -}}
   <
{{- 45}}

formats as 23<45.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jun 29, 2017

@DSanthosh, you're commenting on a closed issue. We only use the issue tracker for bug reports, and don't track things once closed.

For questions about Go, see https://golang.org/wiki/Questions.

@golang golang locked and limited conversation to collaborators Jun 29, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.