cmd/compile: why are stack frames so large?

[rsc]

In src/cmd/go, scanPackage has a 1816-byte stack frame on my 64-bit Mac.
Quickly counting local variables I see about 46 words worth of automatics,
so 368 bytes.  Where is the rest?

[rsc]

Comment 1:

Labels changed: added performance.

[gopherbot]

Comment 2 by dcampbell24:

cd $GOROOT/src/cmd/go
grep -i scanPackage *
returns nothing. I also tried grepping the rest of src and could not find any reference
to scanPackage. Where is scanPackage?

[rsc]

Comment 3:

The code has moved around a bit but the stack frames are probably
still larger than I would expect. The easiest thing to do is to use hg
to extract the code from Jan 18 or so.

[minux]

Comment 4:

perhaps a smaller test case could be:
package main
func f() {
    if false {
        print([]byte{})
    }
}
6g -S:
--- prog list "f" ---
0000 (test.go:3) TEXT    f+0(SB),$16-0
0001 (test.go:7) RET     ,
i think the reason is that we maintain stack frame size as a global variable that
never decreases. maybe we can maintain per-Node (per-Prog) stack requirement
so that if we optimize out some Node/Prog, we can use smaller stack frame size?

[nigeltao]

Comment 5:

One non-obvious source of stack requirements is calling variadic functions, such as
fmt.Println. Even if i is an existing interface{} variable, calling fmt.Println(i)
allocates (on the stack) a backing array, derives a slice from the array, and passes the
slice to fmt.Println.
fmt.Println(i)
is essentially
a := [1]interface{}{i}
s := a[:]
fmt.Println(s...)
Thus, this simple program's f1 function requires an 80-byte stack, despite not
explicitly declaring any local variables:
$ cat main.go
package main
import "fmt"
var i interface{}
func f1() {
    fmt.Println(i)
}
$ go tool 6g -S -o /dev/null main.go | grep TEXT.*f1
0000 (main.go:7) TEXT    f1+0(SB),$80-0
In detail: the 80 bytes (10 quad-words) consist of 40 bytes for implicit locals, and 40
bytes for the fmt.Println call (the 6g calling convention is that callers are
responsible for the stack space for arguments and return values).
The 40 bytes of implicit locals are 16 bytes for the array of 1 element (an interface{}
requires 16 bytes), 16 bytes for the slice (8 byte pointer, 4 byte len, 4 byte cap), and
IIUC, 8 bytes to spill a register to the stack during the implied construction of s.
That register spill is probably a compiler bug.
The 40 bytes of calling-fmt.Println stack is straightforward to explain. The signature
is:
func Println(a ...interface{}) (n int, err error)
which needs 16 bytes for the implicit []interface{} argument, 8 bytes for the int return
value (well, 4 bytes, but each return value is 8-byte aligned) and 16 bytes for the
error return value.
It's an obvious waste that the implicit and short-lived s needs 16 bytes as a local
variable plus 16 bytes as a call argument. That's probably another compiler bug.
Another aspect is that the stack space for the implied array, slice and spill do not
seem to be re-used across repeated calls to variadic functions. Fixing this may be
non-trivial to do correctly if the callee can save the implied slice, or the address of
an implied element, past the lifetime of the call. Still:
$ cat main.go
package main
import "fmt"
var i interface{}
func f1() {
    fmt.Println(i)
}
func f2() {
    fmt.Println(i)
    fmt.Println(i)
}
func f3() {
    fmt.Println(i)
    fmt.Println(i)
    fmt.Println(i)
}
$ go tool 6g -S -o /dev/null main.go | grep TEXT.*f
0000 (main.go:7) TEXT    f1+0(SB),$80-0
0026 (main.go:11) TEXT    f2+0(SB),$120-0
0076 (main.go:16) TEXT    f3+0(SB),$160-0

[nigeltao]

Comment 6:

FYI, the biggest stacks in src/cmd/go are currently:
(build.go:1489) TEXT    (*builder).cgo+0(SB),$2928-112
(test.go:372) TEXT    (*builder).test+0(SB),$1936-56
(build.go:619) TEXT    (*builder).build+0(SB),$1816-32
(test.go:206) TEXT    runTest+0(SB),$1528-24
(pkg.go:299) TEXT    (*Package).load+0(SB),$1360-48
(clean.go:105) TEXT    clean+0(SB),$1296-8
(vcs.go:422) TEXT    repoRootForImportDynamic+0(SB),$1136-40
(get.go:226) TEXT    downloadPackage+0(SB),$1000-24
(get.go:140) TEXT    download+0(SB),$936-24
(test.go:570) TEXT    (*builder).runTest+0(SB),$920-32
(build.go:1378) TEXT    gccgcToolchain.ld+0(SB),$880-96
(build.go:350) TEXT    goFilesPackage+0(SB),$872-24
(build.go:1677) TEXT    (*builder).swigOne+0(SB),$776-104
(pkg.go:474) TEXT    isStale+0(SB),$744-24
(env.go:68) TEXT    runEnv+0(SB),$632-24
(pkg.go:578) TEXT    loadPackage+0(SB),$592-32
(vcs.go:355) TEXT    repoRootForImportPathStatic+0(SB),$480-56
(build.go:1324) TEXT    gcToolchain.cc+0(SB),$472-80
(tool.go:56) TEXT    runTool+0(SB),$448-24
(build.go:1412) TEXT    gccgcToolchain.cc+0(SB),$448-80

[rsc]

Comment 7:

Labels changed: added go1.1maybe.

[rsc]

Comment 8:

[The time for maybe has passed.]

Labels changed: removed go1.1maybe.

[rsc]

Comment 9:

Labels changed: added go1.3.

[rsc]

Comment 10:

https://golang.org/cl/12829043 should take care of the biggest cause.

[robpike]

Comment 11:

Labels changed: removed go1.3.

[rsc]

Comment 12:

Labels changed: added go1.3maybe.

[rsc]

Comment 13:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 14:

Labels changed: added repo-main.