x/tools/cmd/goyacc: When growing the stack, save it in the parser

[rillig]

1. What version of Go are you using (go version)?

go version go1.6.2 windows/amd64

1. What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\rillig\go
set GORACE=
set GOROOT=C:\Program Files (x86)\Go1.6.2
set GOTOOLDIR=C:\Program Files (x86)\Go1.6.2\pkg\tool\windows_amd64
set GO15VENDOREXPERIMENT=1
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1

1. What did you do?

I wrote a Yacc parser for parsing shell commands into an AST.
I accessed the resulting AST via yyParserImpl.stack[1].
For small examples, this worked well.
More complicated examples resulted in nil being on the stack.

1. What did you expect to see?

yyParserImpl.stack[1] should always contain the result, no matter how deeply nested the grammar is.

1. What did you see instead?

When the Yacc parser needs more stack, it grows it. The code is after the yystack: label. This new stack is then local to the current function, and it is not written back into the parser. This means that calling Parse multiple times will create unnecessary copies of the stack. And, more importantly, accessing the actual stack becomes impossible.

References: http://stackoverflow.com/questions/36822702/accessing-result-of-a-golang-yacc-generated-parser

[ianlancetaylor]

It would help a great deal to have a test case showing the problem. Thanks.

[rillig]

issue16163.zip

The grammar implements a simple counter. Some grammar productions need a stack longer than the default initial size. Starting with 16, a new stack gets allocated, which then results in unexpected values being stored in stack[1].

[gopherbot]

Change https://go.dev/cl/424776 mentions this issue: cmd/goyacc: save stack in parser after resizing