go/parser: more robust error handling

[rsc]

I'm going to try to start filing bugs against go/parser when I notice things that trigger less than ideal error messages in the course of ordinary usage (via goimports, go test, and so on), because I think there are some simple cases that can be improved. I expect most of the initial ones will be about getting too many errors. Here's the first one.

package p

func bad() {
    if f()) {
        return
    }
}

func F1() {}
func F2() {}
func F3() {}
func F4() {}
func F5() {}
func F6() {}
func F7() {}
func F8() {}
func F9() {}
func F10() {}

This produces:

$ gofmt /tmp/xx.go
/tmp/xx.go:4:8: expected '{', found ')'
/tmp/xx.go:9:6: expected '(', found 'IDENT' F1
/tmp/xx.go:10:6: expected '(', found 'IDENT' F2
/tmp/xx.go:11:6: expected '(', found 'IDENT' F3
/tmp/xx.go:12:6: expected '(', found 'IDENT' F4
/tmp/xx.go:13:6: expected '(', found 'IDENT' F5
/tmp/xx.go:14:6: expected '(', found 'IDENT' F6
/tmp/xx.go:15:6: expected '(', found 'IDENT' F7
/tmp/xx.go:16:6: expected '(', found 'IDENT' F8
/tmp/xx.go:17:6: expected '(', found 'IDENT' F9
/tmp/xx.go:18:6: expected '(', found 'IDENT' F10
$ 

This is a reduced version of a real case I had, where an extra paren in a function near the top of a long source file triggered many many messages. In this program, the braces are still balanced, so there should be a good signal for the parser to recover by the end of the function (and ideally the end of the enclosing block, although here they are the same).

This is a case where the yacc grammar's error recovery is actually doing better:

$ go tool compile /tmp/xx.go
/tmp/xx.go:4: syntax error: unexpected ), expecting {
/tmp/xx.go:7: syntax error: unexpected }
$

Also the IDENT in the go/parser error is probably not helpful to the user. It's an internal detail.

[griesemer]

FWIW, the new syntax parser and thus cmd/compile produce exactly one error in this case:

x.go:6:11: syntax error: unexpected ), expecting semicolon or newline

[gopherbot]

Change https://golang.org/cl/87495 mentions this issue: go/parser: more robust error handling for 'if' headers

[gopherbot]

Change https://golang.org/cl/87900 mentions this issue: go/parser: improved error message for unexpected literals