cmd/gofmt: applying gofmt to partial programs changes raw string literal values

[griesemer]

$ cat x.go
    func f() {
        const s = `
foo
`
        println(s)
    }

$ cat x.go | gofmt
    func f() {
        const s = `
    foo
    `
        println(s)
    }

After gofmt, the raw string s is changed: it shouldn't be indented like the rest of the
code.

[griesemer]

Comment 1:

A simpler example:
$ cat x.go
    const s = `
foo
`
$ cat x.go | gofmt
    const s = `
    foo
    `

[rsc]

Comment 2:

Labels changed: added priority-later, removed priority-triage.

[rsc]

Comment 3:

I think it's too late to be changing gofmt right now. We're going to have to keep living
with this bug (so far it doesn't seem to have caused much trouble).

Labels changed: added go1.2, removed go1.1.

[dsymonds]

Comment 4:

There doesn't seem time or interest for this making it into Go1.2.

Labels changed: removed go1.2.

[rsc]

Comment 5:

Labels changed: added go1.3maybe.

[rsc]

Comment 6:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 7:

Labels changed: added repo-main.

[dmitshur]

Comment 8:

I've been working on fixing this bug, but in the process I've realized it may not be a
bug. Consider the following.
Note that the issue can only occur under these 2 conditions:
- Applying gofmt to a partial Go program.
- The partial Go code has some indent (i.e. first non-blank line has leading whitespace).
Now, consider the context where the above may happen. It could happen when you're trying
to include some partial Go code in the docs/comments for a func.
For example,
//  if  err!=nil      {
//      source:=    strings.NewReader("line 1.\nline 2.\n")
//      return source
//  }
It may make sense to try and apply gofmt to that indented partial Go code (by taking the
"commented out text" value, which does not include the // comment symbols), which should
make it look like this:
//  if err != nil {
//      source := strings.NewReader("line 1.\nline 2.\n")
//      return source
//  }
Now consider what the equivalent comment should look like if it used a raw string
literal (with the same value). I would argue it makes the most sense to do:
//  if err != nil {
//      source := strings.NewReader(`line 1.
//  line 2.
//  `)
//      return source
//  }
I would argue that it makes most sense. When godoc parses it, it displays the code
correctly, see http://godoc.org/github.com/shurcooL/play/54#Issue4449Sample1.
Consider the alternatives:
//  if err != nil {
//      source := strings.NewReader(`line 1.
//line 2.
//`)
//      return source
//  }
//  if err != nil {
//      source := strings.NewReader(`line 1.
line 2.
`)
//      return source
//  }
The first one would no longer be parsed by godoc such that the code segment is
considered a single code segment (see
http://godoc.org/github.com/shurcooL/play/54#Issue4449Sample2). The second alternative
would render the original full Go program with the comment to be invalid, since a part
of the comment stops being a comment when // is removed (see
https://github.com/shurcooL/play/blob/90362c11c0d5560ec7eca380c2b6b062793a355c/54/main.go#L25-L40).
So, I think the best way to go forward is to not consider the leading indent in front of
partial Go code should to be a part of said partial Go code. If that's the case, then
the current gofmt behavior is correct - it does _not_ change the raw string literal
values. However, under that perspective, the current go/format behavior would become
incorrect, because it does change raw string literal value of partial indented Go code
(but that's a separate issue).

[dmitshur]

Comment 9:

Although, I've just realized the current indent behavior cannot be correct either,
because it's not idempotent.
$ cat x.go
    func f() {
        const s = `
foo
`
        println(s)
    }
$ cat x.go | gofmt
    func f() {
        const s = `
    foo
    `
        println(s)
    }
$ cat x.go | gofmt | gofmt | gofmt | gofmt
    func f() {
        const s = `
                foo
                `
        println(s)
    }
Sigh. Trying to have "correct" behavior of preserving leading whitespace is really
tricky and I wish it was handled by external tools rather than gofmt itself. I don't see
a good solution now. Even if you try to strip the leading whitespace when parsing
partial Go code (to fix the lack of idempotence), what happens if some of the lines do
not contain the same leading whitespace? The value of the raw string literal in code
like this will become ambiguous:
    func f() {
        const s = `
    foo
`
        println(s)
    }

[griesemer]

Comment 10:

Thank you for investigating the issue and uncovering the various problem scenarios (and
welcome to the complicated world of gofmt...:-).
I think the common case for partial program gofmt-ing is application inside an editor
(emacs, etc.): A selected piece of code should be gofmt-ed; it's likely for that code to
be indented because it's inside a function. In that case it shouldn't modify raw
strings. The case of raw strings inside documentation comments seems much less likely.
So I'd consider this still a bug.

[dmitshur]

Comment 11:

Fair enough, I'll continue to consider this a bug.
Just an update here, I have code that fixes this and 5551 issues, but I haven't gotten
around to making a CL out of it yet. You can see it here:
https://github.com/shurcooL/go/compare/0f4891c007a...golang-issues-5551-4449
The code works and does the job (and all tests pass), but it's still very unclean and
needs to be improved. I hope to do that as soon as possible...
> I think the common case for partial program gofmt-ing is application inside an editor
(emacs, etc.):
> A selected piece of code should be gofmt-ed; it's likely for that code to be indented
because it's
> inside a function. In that case it shouldn't modify raw strings.
I just want to comment that sounds very unusual. I see no reason why anyone would want
to select and gofmt a part of a .go file instead of just having an on-save hook that
runs either gofmt or goimports on the entire file (in emacs, vim, Sublime Text or any
other editor). What would happen if you select a partial for statement, or start
selection not from the beginning of a line? Why not gofmt the entire .go file?

[griesemer]

Comment 12:

Re: your last comment in #11: It's not as unusual as it may seem; that's the reason why
gofmt has so much (pretty ugly) machinery to handle this case. One reason might be that
there's only a partial package file present, or that parts of the file are syntactically
incomplete.

[griesemer]

Comment 13:

Re: your last comment in #11: It's not as unusual as it may seem; that's the reason why
gofmt has so much (pretty ugly) machinery to handle this case. One reason might be that
there's only a partial package file present, or that parts of the file are syntactically
incomplete.

[gopherbot]

Comment 14:

CL https://golang.org/cl/142360043 mentions this issue.

[griesemer]

Comment 15:

This issue was closed by revision 912ec19.

Status changed to Fixed.