cmd/compile: unneeded stack checks and stack handling

[marigonzes]

What version of Go are you using (go version)?

$ go version
go version devel +24b4301 Wed May 22 02:10:36 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What did you do?

I compiled the following program https://play.golang.org/p/STj160cWl4l, simplified from a bigger code base.

What did you expect to see?

I expected function a1 to be compiled to a simple return, without the need for the stack checks and stack handling after a2 returns, given that a2 is compiled to a simple return.
My expectation came from the fact that in this other program (https://play.golang.org/p/VBBQJvevHxw), in which a2 still compiles to the same code, a1 doesn't have the stack checks/stack handling.

What did you see instead?

Instead, a1 was compiled with stack checks/stack handling.

[randall77]

The reason for this discrepancy is that in your second example, a2 is inlined into a1. a1 is then a leaf with a small stack frame and requires no stack check. In your first example, a2 is not inlined because at inlining time, the body of a2 looked too big to inline. So a1 is no longer a leaf and requires at least some SP manipulation.

I'm surprised it has a stack check though. I thought we elided the stack checks if the call subtree all fit in the guard area. That may be the bug here.

[cherrymui]

Currently, the stack bound check elimination is not interprocedural. We assemble one function at a time, and we don't look at the call graph. Currently we only eliminate stack bound check for leaf-ish functions (allowing calls to some special functions like duffzero or some panic functions).

[cherrymui]

The linker computes the call graph, and check that the stack usage of nosplit functions doesn't go beyond the stack guard. We could imagine something in the linker that after the call graph computation, if it sees a subtree doesn't use much stack it can change the call target to skip over the stack bound check prologue (and/or change the prologue to NOPs).