runtime: unexpected reuse of memory when debugging

[hao-liangyou]

Go version

go version go1.24.1 linux/amd64

Output of go env in your module/workspace:

should be irrelevant

What did you do?

1. start debugging in VSCode with the code
2. let it run until it automatically stops at panic (w/o breakpoint)
3. inspect the local variable s

What did you see happen?

Sometimes its value is the expected 0, 1, 2, 3, ....
Sometimes its value is the unexpected MaxUint, 0, MaxUint, 0, .... (It looks like the memory of s is reused in the deferred function)
Occasionally its value is meaningless.

What did you expect to see?

Its value is always 0, 1, 2, 3, ...

[randall77]

Kind of an odd behavior, but not too terribly surprising.

Are you compiling with -N or -l? (VSCode might set those automatically? I'm not sure.)

At the panic call, s is dead, so it's not terribly surprising that its backing store is reused by something else. I don't think we guarantee anything here.

It does look tricky though, as the backing store for s is on the stack, but all the places you're writing maxint,0,.. to are on the heap. So probably what is happening here is that the stack is copied (freeing the old backing store for s), then one of the allocations in the defer to which you write maxint,0,... happens to use the same memory.
The fmt.Printf is probably causing the stack use that causes the stack copy, which is why it is required.

[hao-liangyou]

Are you compiling with -N or -l? (VSCode might set those automatically? I'm not sure.)

yes, it set -gcflags all=-N -l automatically

At the panic call, s is dead, so it's not terribly surprising that its backing store is reused by something else. I don't think we guarantee anything here.

I'm not sure, s won't be used in the code anymore, but should still exist somewhere for inspection by debugger?

New observation:
if fmt.Printf("s: %v\n", s[rand.IntN(len(s))]) at the end of the defer, it is always expected, but not in the debugger.
if fmt.Printf("s: %v\n", s) at the end of the defer, both are always expected.

[randall77]

I'm not sure, s won't be used in the code anymore, but should still exist somewhere for inspection by debugger?

We generally don't want to change the lifetime of variables when debugging. That would affect the GC behavior of the program, which is probably not what you want when trying to reproduce a problem under the debugger.

if fmt.Printf("s: %v\n", s[rand.IntN(len(s))]) at the end of the defer, it is always expected, but not in the debugger.
if fmt.Printf("s: %v\n", s) at the end of the defer, both are always expected.

That makes sense, using only the length means the pointer to the backing slice is dead at the panic point. Using the whole thing means the pointer is still live.

I think there could be some help from the debugger here to let you know that the variable you're trying to inspect is dead/stale, kind of like how we print ? in tracebacks in similar situations.

[hao-liangyou]

if fmt.Printf("s: %v\n", s[rand.IntN(len(s))]) at the end of the defer, it is always expected, but not in the debugger.
if fmt.Printf("s: %v\n", s) at the end of the defer, both are always expected.

That makes sense, using only the length means the pointer to the backing slice is dead at the panic point. Using the whole thing means the pointer is still live.

In fmt.Printf("s: %v\n", s[rand.IntN(len(s))]), there is access to s, not only len. I think the slice should not be dead until defer returns?

[randall77]

That's a good point, I missed that s is also being indexed.

In your original code, s is not even in scope in the defer. Could you post the exact code that demonstrates this case?

[hao-liangyou]

That's a good point, I missed that s is also being indexed.

In your original code, s is not even in scope in the defer. Could you post the exact code that demonstrates this case?

here

just move s := make( to the begin of main, add fmt.Printf at the end of the defer, and extra random to ensure no optimization

[randall77]

I think I see what is going on here. Delve stops at the panic line after having run all the defers. So code in the deferred function that accesses s is not enough by itself to keep s alive.
What does suppress the bug here is a reference in the deferred function that escapes s. Then s is allocated in the heap instead of the stack and keeps this bug from happening (s is still dead, so the same sort of bug could happen, but is distinctly less likely).

[randall77]

Here's a simplified, import-free reproducer:

package main

func main() {
	s := make([]int, 128)
	for i := range s {
		s[i] = i
	}

	defer func() {
		for i := range 64 * 1024 {
			useStack(1000)
			p := make([]int, 1<<(i%10))
			for i := range p {
				p[i] = len(p)
			}

		}
		println(s[0])
	}()

	panic("")
}

func useStack(n int) {
	if n == 0 {
		return
	}
	useStack(n - 1)
}

Run

dlv debug issue73117.go
continue
up
up
print s

It should print []int len: 128, cap: 128, [0,1,2,3,4,5,.... On error it prints []int len: 128, cap: 128, [256,256,256,256,... (or maybe 128 or 512 as the number instead).

[randall77]

@aarzilli @derekparker Not sure if there is really anything to do here, but perhaps dlv could warn when trying to print a dead variable that its contents might not be correct?
Might also be worth checking that the DWARF is correct for this case.

[aarzilli]

Might also be worth checking that the DWARF is correct for this case.

It does look like it is, the DW_AT_location for s is just DW_OP_fbreg -0x28, it looks fine.

Not sure if there is really anything to do here, but perhaps dlv could warn when trying to print a dead variable that its contents might not be correct?

We have no indication of this in dwarf and so far we have refrained from digging through the GC data due to concerns with maintainability (hard to keep it working, hard to notice if something changes from a version to another). If we can figure out a way to do this that's easy (ish) to keep working, we could do it.

Other than that I don't think we can do anything about this (the other solution, extending the lifetimes to match the scope when optimizations are disabled, has already been dismissed because it creates its own set of problems).

[randall77]

Ok, then I think we can close this as unfortunate but not feasibly fixable.