cmd/compile: missing DWARF location lists for return values

[michael-obermueller]

What version of Go are you using (go version)?

$ go version
go version go1.18beta2 windows/amd64

Does this issue reproduce with the latest release?

With Go1.17.6, DWARF entries for return values of optimized functions were not available at all due to related issue #48573

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\...\AppData\Local\go-build
set GOENV=C:\Users\...\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\...\go\pkg\mod
set GONOPROXY=...
set GONOSUMDB=...
set GOOS=windows
set GOPATH=C:\Users\...\go
set GOPRIVATE=...
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Users\...\sdk\go1.18beta2
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Users\...\sdk\go1.18beta2\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.18beta2
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\workspaces\test\go.mod
set GOWORK=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\...\AppData\Local\Temp\go-build1425766270=/tmp/go-build -gno-record-gcc-switches
GOROOT/bin/go version: go version go1.18beta2 windows/amd64
GOROOT/bin/go tool compile -V: compile version go1.18beta2

What did you do?

Build an application with the Go 1.18beta2 toolchain using the command go build Main.go (optimized binary):

package main

func main() {
	_, _, _ = testFunc(1,2,3)
}

//go:noinline
func testFunc(a, b, c int) (d int, e int, f int) {
	return c, b, a
}

Disassembly of `main.testFunc`

0000000000459d40 <main.testFunc>:
  459d40:	48 89 c2          mov    %rax,%rdx
  459d43:	48 89 c8          mov    %rcx,%rax
  459d46:	48 89 d1          mov    %rdx,%rcx
  459d49:	c3                   	retq 

What did you expect to see?

DWARF location lists for function return values:

Example for return value `d`

 <2><3f794>: Abbrev Number: 17 (DW_TAG_formal_parameter)
    <3f795>   DW_AT_name        : d
    <3f797>   DW_AT_variable_parameter: 1
    <3f798>   DW_AT_decl_line   : 8
    <3f799>   DW_AT_type        : <0x40055>
    <3f79d>   DW_AT_location    : 0x<loclist addr> (location list)
...
    <loclist addr> ffffffffffffffff 0000000000459d40 (base address)
    <loclist addr> <0000000000459d49> <0000000000459d49> (DW_OP_reg0 (rax))

What did you see instead?

Missing DWARF location lists for all function return values of main.testFunc:

DWARF data of main.testFunc

debug.txt

<1><3f747>: Abbrev Number: 3 (DW_TAG_subprogram)
    <3f748>   DW_AT_name        : main.testFunc
    <3f756>   DW_AT_low_pc      : 0x459d40
    <3f75e>   DW_AT_high_pc     : 0x459d4a
    <3f766>   DW_AT_frame_base  : 1 byte block: 9c 	(DW_OP_call_frame_cfa)
    <3f768>   DW_AT_decl_file   : 0x2
    <3f76c>   DW_AT_external    : 1
 <2><3f76d>: Abbrev Number: 18 (DW_TAG_formal_parameter)
    <3f76e>   DW_AT_name        : a
    <3f770>   DW_AT_variable_parameter: 0
    <3f771>   DW_AT_decl_line   : 8
    <3f772>   DW_AT_type        : <0x40055>
    <3f776>   DW_AT_location    : 0x68d4f (location list)
 <2><3f77a>: Abbrev Number: 18 (DW_TAG_formal_parameter)
    <3f77b>   DW_AT_name        : b
    <3f77d>   DW_AT_variable_parameter: 0
    <3f77e>   DW_AT_decl_line   : 8
    <3f77f>   DW_AT_type        : <0x40055>
    <3f783>   DW_AT_location    : 0x68d82 (location list)
 <2><3f787>: Abbrev Number: 18 (DW_TAG_formal_parameter)
    <3f788>   DW_AT_name        : c
    <3f78a>   DW_AT_variable_parameter: 0
    <3f78b>   DW_AT_decl_line   : 8
    <3f78c>   DW_AT_type        : <0x40055>
    <3f790>   DW_AT_location    : 0x68db5 (location list)
 <2><3f794>: Abbrev Number: 17 (DW_TAG_formal_parameter)
    <3f795>   DW_AT_name        : d
    <3f797>   DW_AT_variable_parameter: 1
    <3f798>   DW_AT_decl_line   : 8
    <3f799>   DW_AT_type        : <0x40055>
    <3f79d>   DW_AT_location    : 0 byte block: 	()
 <2><3f79e>: Abbrev Number: 17 (DW_TAG_formal_parameter)
    <3f79f>   DW_AT_name        : e
    <3f7a1>   DW_AT_variable_parameter: 1
    <3f7a2>   DW_AT_decl_line   : 8
    <3f7a3>   DW_AT_type        : <0x40055>
    <3f7a7>   DW_AT_location    : 0 byte block: 	()
 <2><3f7a8>: Abbrev Number: 17 (DW_TAG_formal_parameter)
    <3f7a9>   DW_AT_name        : f
    <3f7ab>   DW_AT_variable_parameter: 1
    <3f7ac>   DW_AT_decl_line   : 8
    <3f7ad>   DW_AT_type        : <0x40055>
    <3f7b1>   DW_AT_location    : 0 byte block: 	()

I have checked many functions of optimized binaries, for all of them I can see the same problem.
Thanks

[ianlancetaylor]

CC @thanm

[thanm]

I took a look. I think things are working mostly as intended, within the limitations we have as part of our compiler design and how things work with optimization turned on.

The DWARF variable location expressions emitted by the compiler when optimization is on are basically "best effort" at the moment -- we do our best to emit correct DWARF when we can, but there are going to be cases where the optimizer will confuse the DWARF emitter and it can't generate anything reasonable (or anything at all), and other cases where we miss variables completely.

The incoming Go code looks like:

L7:  //go:noinline
L8:  func testFunc(a, b, c int) (d int, e int, f int) {
L9:  	return c, b, a
L10: }

The assembly we generate looks like this:

00000  (8) TEXT "".testFunc(SB), ABIInternal
00006 (+9) MOVQ AX, DX
00007  (9) MOVQ CX, AX
00008  (9) MOVQ DX, CX
00009  (9) RET

The "+" tag on line (+9) on the first instruction indicates that this spot is marked in the DWARF line table as being the start of a statement (hence this is where the debugger will stop).

Semantics for Go return parameters are that they initially have a zero value, then take on other values if assigned to by name or by a return statement. In this case at line 9, the return hasn't yet executed, so there is no specific location we can point to (register or memory) that contains the value we need for a variable like "d".

There are compilers out there that will recognize portions of the lifetime of a variable that are constant, and emit multi-range location expressions where some of the ranges hold locations (register or stack) and other ranges hold constants (GCC does a pretty good job of this). The Go compiler currently doesn't try to do this, since it adds complexity to the value tracking machinery. If it can't see a specific register or chunk of memory that holds the value of a named variable, it won't emit a debug location for it.

It's true that once the "MOVQ CX, AX" has executed, then we have a concrete location for "d", so we could generate a location list that started at that spot. It's not clear how useful that would be, however, since the only way to get to that spot is to step instruction by instruction (something most users of debuggers are not going to do).

For return parameters in general I think the Go compiler has always done a pretty substandard job, even with the register ABI. Prior to the register ABI work, it would simply put out a location description pointing the stack location and be done with it. So for a function like

//go:noinline
 func testFunc(a, b, c int) (d int, e int, f int) {
   d = 101
   d += G          /* SPOT1 */
   println(d)      /* SPOT2 */
   return c, b, a
 }
 var G int

If you were to stop on SPOT1 or SPOT2 above, printing "d" would just show an incorrect value of zero.

When the register ABI work was completed, there were a lot more opportunities during late call lowing for return params to get "lost in the sauce", which is what's happening now in many cases.

So just to summarize: yes, we could definitely be doing better in terms of emitting DWARF location expressions for return params, but fixing this is something that would require some concerted design and implementation work, it isn't a case where we could resolve the issue with small tweaks to the existing code.