cmd/compile: invalid DWARF location list for local var - erroneously extends to function epilogue

[andreimatei]

What version of Go are you using (go version)?

1.20.3

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/andrei/.cache/go-build"
GOENV="/home/andrei/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/andrei/work/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/andrei/work"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/andrei/sdk/go1.19.9"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/andrei/sdk/go1.19.9/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.9"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3969983697=/tmp/go-build -gno-record-gcc-switches"

What did you do?

The DWARF location list for some local variables (vars declared on the first line within a function?) appears to be incorrect: it extends to include code placed in the object file at the end of the function that has to do with growing the stack (i.e. calling runtime.morestack_noctxt). Even though it's placed at the end of the function, this code executes when the function start if it executes at all (i.e. when the stack actually has to grow). By including the program counters corresponding to this code in the variable's location list, the compiler is declaring that the variable is live at those locations. In fact, the variable is not live; when the respective code executes, the variable has yet to be initialized. The respective location list instructions are thus invalid - where they do be followed to read the variable, we'd read random heap memory.

Let's study an example.
Consider the variable localVar in following program:
https://go.dev/play/p/P4_7_JfCbzo

package main

var global int

//go:noinline
func foo() {
	// localVar's loclist erroneously extends to the end of foo's code -
	// including the epilogue.
	var localVar int
	localVar = global
	localVar++

	// Force the stack to grow when foo() starts executing. This isn't necessary
	// for reproducing the issue, but it is useful if trying to actually
	// demonstrate that the bug is "exploitable" by trapping on the stack
	// expansion and attempting to read the variable.
	var xx [][5000]byte
	var x1 [5000]byte
	xx = append(xx, x1)

	global = localVar
}

func main() {
	global = 42
	foo()
}

This is the debug information produced for localVar, with my arrow markings:

$ llvm-dwarfdump-16 --debug-info --color  --name=main.foo --show-children main                          

0x000640ab: DW_TAG_subprogram
              DW_AT_name        ("main.foo")
              DW_AT_low_pc      (0x0000000000457240)
---->         DW_AT_high_pc     (0x00000000004572d1)
              DW_AT_frame_base  (DW_OP_call_frame_cfa)
              DW_AT_decl_file   ("/home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go")
              DW_AT_decl_line   (6)
              DW_AT_external    (0x01)

0x000640cd:   DW_TAG_variable
                DW_AT_name      ("localVar")
                DW_AT_decl_line (9)
                DW_AT_type      (0x000000000004cf9c "int")
                DW_AT_location  (0x0007356d: 
                   [0x0000000000457264, 0x0000000000457299): DW_OP_reg1 RDX
---->              [0x0000000000457299, 0x00000000004572d1): DW_OP_fbreg -5024)

Notice that the variable has a loclist extending to the end of the function's code - 0x00000000004572d1. The info says that the variable can be read from the stack.

By disassembling, we can see that this is incorrect. The last instructions before 0x4572d1 are about the stack growth. When that code executes, the variable is not alive, and attempting to read it according to the generated loclist would result in reading essentially random heap memory (outside of the current function's stack frame).

Disassembly

$  llvm-objdump-16 --disassemble --line-numbers --source --disassemble-symbols=main.foo  main

0000000000457240 <main.foo>:
; main.foo():
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:6
; func foo() {
  457240: 49 89 e4                     	movq	%rsp, %r12
  457243: 49 81 ec 58 13 00 00         	subq	$0x1358, %r12           # imm = 0x1358
  45724a: 72 7b                        	jb	0x4572c7 <main.foo+0x87>
  45724c: 4d 3b 66 10                  	cmpq	0x10(%r14), %r12
  457250: 76 75                        	jbe	0x4572c7 <main.foo+0x87>
  457252: 55                           	pushq	%rbp
  457253: 48 89 e5                     	movq	%rsp, %rbp
  457256: 48 81 ec d0 13 00 00         	subq	$0x13d0, %rsp           # imm = 0x13D0
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:10
; 	localVar = global
  45725d: 48 8b 15 fc 03 0a 00         	movq	0xa03fc(%rip), %rdx     # 0x4f7660 <main.global>
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:18
; 	var x1 [5000]byte
  457264: 44 0f 11 7c 24 48            	movups	%xmm15, 0x48(%rsp)
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:11
; 	localVar++
  45726a: 48 ff c2                     	incq	%rdx
  45726d: 48 89 54 24 40               	movq	%rdx, 0x40(%rsp)
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:18
; 	var x1 [5000]byte
  457272: 48 8d 7c 24 50               	leaq	0x50(%rsp), %rdi
  457277: b9 70 02 00 00               	movl	$0x270, %ecx            # imm = 0x270
  45727c: 31 c0                        	xorl	%eax, %eax
  45727e: f3 48 ab                     	rep		stosq	%rax, %es:(%rdi)
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:19
; 	xx = append(xx, x1)
  457281: 31 c0                        	xorl	%eax, %eax
  457283: bb 01 00 00 00               	movl	$0x1, %ebx
  457288: 31 c9                        	xorl	%ecx, %ecx
  45728a: 48 89 df                     	movq	%rbx, %rdi
  45728d: 48 8d 35 2c 5c 00 00         	leaq	0x5c2c(%rip), %rsi      # 0x45cec0 <type:*+0x4ec0>
  457294: e8 07 a5 fe ff               	callq	0x4417a0 <runtime.growslice>
  457299: 48 8b 54 24 48               	movq	0x48(%rsp), %rdx
  45729e: 48 89 10                     	movq	%rdx, (%rax)
  4572a1: 48 8d 78 08                  	leaq	0x8(%rax), %rdi
  4572a5: 48 8d 74 24 50               	leaq	0x50(%rsp), %rsi
  4572aa: b9 70 02 00 00               	movl	$0x270, %ecx            # imm = 0x270
  4572af: f3 48 a5                     	rep		movsq	(%rsi), %es:(%rdi)
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:21
; 	global = localVar
  4572b2: 48 8b 54 24 40               	movq	0x40(%rsp), %rdx
  4572b7: 48 89 15 a2 03 0a 00         	movq	%rdx, 0xa03a2(%rip)     # 0x4f7660 <main.global>
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:22
; }
  4572be: 48 81 c4 d0 13 00 00         	addq	$0x13d0, %rsp           # imm = 0x13D0
  4572c5: 5d                           	popq	%rbp
  4572c6: c3                           	retq
; /home/andrei/work/src/github.com/andreimatei/play/go-debuginfo-examples/loclist-covering-epilogue/main.go:6
; func foo() {
  4572c7: e8 d4 ce ff ff               	callq	0x4541a0 <runtime.morestack_noctxt.abi0>
  4572cc: e9 6f ff ff ff               	jmp	0x457240 <main.foo>
  4572d1: cc                           	int3

Notice the retq at pc 4572c6. I believe the loclist for localVar should end around there.
Also notice how the function prelude jumps to 0x4572c7 is the stack needs growth.

I have noticed (but I'm not sure) that the loclist extends to the end of the epilogue when the variable is declared at the very beginning of the function and lives until the end of the function. I think function arguments might be susceptible to the same problem.

As a random tidbit of trivia, Delve takes a variable's DW_AT_decl_line into consideration, in addition to its location list, when deciding whether a particular variable can be read at a given pc, for better or worse. This way, Delve avoids using the erroneous instructions in the pc range where they are invalid (because the respective epilogue code is marked as corresponding to the line on which the function is declared, which is below the variable's decl_line). Function arguments are treated differently by Delve; I think those might be susceptible to invalid reads (but I'm not sure).

What did you expect to see?

I expected the location list for localVar to not extend past the instruction returning from the function.

What did you see instead?

The location list for localVar extends too much.

---

If my involvement would be useful, I'd be happy to work on addressing this bug myself - if someone knows the solution and is willing to guide me.

cc @dr2chase @thanm @heschi as people active on go's debug symbol generation
cc @aarzilli for the Delve connection

[randall77]

Kind of a novice in this area, but the compiler makes the location lists, and the wrapper for stack overflow is not added until the compiler (or at least the SSA part, which builds the location lists) is done. Maybe the compiler is using some sort of "all-the-way-to-the-end" marker?
The wrapping is done by cmd/internal/obj/x86/obj6.go:stacksplit, which is called in cmd/compile/internal/pgen.go:Compile. The Flush call calls stacksplit, which is after genssa is done.
That's about the limit of my understanding in this area.

[andreimatei]

Thank you for the hints; they helped.

Maybe the compiler is using some sort of "all-the-way-to-the-end" marker?

Yeah, something like this was going on. I believe I have a patch that makes the "the end" marker be interpreted correctly. I will work on sending it over.