Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/parser: eats \r in comments #11151

Closed
dvyukov opened this issue Jun 10, 2015 · 4 comments
Closed

go/parser: eats \r in comments #11151

dvyukov opened this issue Jun 10, 2015 · 4 comments
Assignees
Milestone

Comments

@dvyukov
Copy link
Member

@dvyukov dvyukov commented Jun 10, 2015

In the following program go/parser/printer turn a correct program into incorrect one, and so the second parsing fails:

package main

import (
    "bytes"
    "fmt"
    "go/parser"
    "go/printer"
    "go/token"
)

func main() {
    data := []byte("package A   /**\r/*/\n")
    fset := token.NewFileSet()
    f, err := parser.ParseFile(fset, "src.go", data, parser.ParseComments|parser.DeclarationErrors|parser.AllErrors)
    if err != nil {
        return
    }
    buf := new(bytes.Buffer)
    printer.Fprint(buf, fset, f)
    fset1 := token.NewFileSet()
    _, err = parser.ParseFile(fset1, "src.go", buf.Bytes(), parser.ParseComments|parser.DeclarationErrors|parser.AllErrors)
    if err != nil {
        fmt.Printf("source0: %q\n", data)
        fmt.Printf("source1: %q\n", buf.Bytes())
        panic(err)
    }
}
source0: "package A\t/**\r/*/\n"
source1: "package A\t/**/*/\n"
panic: src.go:1:15: expected ';', found '*'

go version devel +b0532a9 Mon Jun 8 05:13:15 2015 +0000 linux/amd64

@dvyukov dvyukov changed the title go/parser: turns correct program into incorrect one go/parser: eats \r in comments Jun 10, 2015
@ianlancetaylor ianlancetaylor added this to the Go1.5Maybe milestone Jun 10, 2015
@griesemer griesemer added the Thinking label Jun 10, 2015
@griesemer griesemer modified the milestones: Unplanned, Go1.5Maybe Jun 10, 2015
@griesemer
Copy link
Contributor

@griesemer griesemer commented Jun 10, 2015

This is non-urgent and extremely unlikely to be a problem in real-world programs.

Background: This is done deliberately, the go/scanner strips \r from comments explicitly ( http://golang.org/cl/6225047 ) - this was done in response to issue #3647 .

That said, perhaps the original change was not ideal or too aggressive, and/or the spec should be clarified (as in \r are stripped from comments).

@griesemer
Copy link
Contributor

@griesemer griesemer commented Sep 12, 2015

  1. The spec should not be changed since comments (and thus their contents) are handled like white space independent of content.
  2. The scanner should probably leave \r in //-style comments but for one followed immediately by \n (the spec considers this "the end of the line" ( http://golang.org/ref/spec#Comments ).
@griesemer
Copy link
Contributor

@griesemer griesemer commented Sep 12, 2015

Alternatively, the scanner may not change comment contents at all, but the printer could apply the fix where needed.

@griesemer griesemer added NeedsFix and removed Thinking labels Jan 12, 2018
@griesemer griesemer modified the milestones: Unplanned, Go1.11 Jan 12, 2018
@gopherbot
Copy link

@gopherbot gopherbot commented Jan 12, 2018

Change https://golang.org/cl/87498 mentions this issue: go/scanner: don't eat \r in comments if that shortens the comment

@gopherbot gopherbot closed this in ea006a8 Feb 12, 2018
@golang golang locked and limited conversation to collaborators Feb 12, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.