runtime: provide a mechanism for debuggers to execute a function in the program being debugged

[komuw]

Background:

I'm new to Go. I've been going through the Go exercises over at; http://exercism.io/languages/go/about.
For some of the exercises in there, I needed a debugger so I came acrosss; https://github.com/derekparker/delve which is really nice.

However I needed to call a function - inside the debugging session- but I was unable to do so. After some google-fu, I came across this issue; https://github.com/derekparker/delve/issues/119 .
That issue ends with the conclusion that for the issue to be fixed, it would need some changes in Go itself.

And then the comment:
maybe we should file an issue in https://github.com/golang/go/issues? We had a conversation with @rsc, and he told us that Go team is going to provide a good basement for debugger at least. - https://github.com/derekparker/delve/issues/119#issuecomment-317032805

Proposal:

Go should provide the good basement for debugger that @rsc alluded to in the comment.

I'm raising this issue here because I asked in the delve issue mentioned above and they never got around to filing an issue with Go and also because, I believe, the floor is now open to discussing potentially backward incompatible things(Go2); some people want generics in Go2, as a new gopher I just want a well integrated debugger.

[dlsniper]

This is far too wide to be a proposal (and it's not even what a proposal should be).

For the specific case at hand, the issue should be: provide a mechanism to execute functions via external debuggers. Even this would then need to be split into different sub-issues since the problem itself is pretty complex, see https://github.com/derekparker/delve/issues/119#issuecomment-317019335

As such, I think this should be closed and a separate issue should be opened for allowing function calls via debuggers.

Regarding the other aspects of debugging, I've seen a lot of CLs being merged that improve various aspects of debugging, such as better scoping information.

[komuw]

@dlsniper I stand corrected.
To make this proposal, I first went through some of the other proposals in this repo(some by the core team).

I honestly have nothing more to offer. I just know the way I would like things to be, as to how to get there; I'll defer to others.

[ianlancetaylor]

Thank you for filing this. A proposal is more helpful when it provides a proposed solution (see https://github.com/golang/proposal/blob/master/README.md). At this point this is more of a bug report, and I think the bug you are reporting is that debuggers need some way to safely execute a Go function. I'll retitle this issue on that assumption.

[komuw]

Thank you so much @ianlancetaylor ! The new title makes much more sense. And thanks for the link to the proposal Documentation

…

On Aug 29, 2017 11:32 PM, "Ian Lance Taylor" ***@***.***> wrote: Thank you for filing this. A proposal is more helpful when it provides a proposed solution (see https://github.com/golang/ proposal/blob/master/README.md). At this point this is more of a bug report, and I think the bug you are reporting is that debuggers need some way to safely execute a Go function. I'll retitle this issue on that assumption. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#21678 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AE7LUUMmS5Okcpj-4R6zG0WENyGN5owIks5sdHVYgaJpZM4PFf7g> .

[heschi]

Spoke with @hyangah, @derekparker, @aarzilli about this; any mistakes are my own.

To me, the most obvious way to call a function from a debugger would be:

• Start with a stopped thread (that hit a breakpoint, presumably) and decide to call a function
• Save all the registers of the stopped thread
• Put the function arguments on the stack
• Set a breakpoint at the end of the called function
• Set the thread's PC to the beginning of the called function
• Resume the thread
• When it hits the end of the called function, restore its state and return control to the user.

There are a number of issues with this approach, but the most critical is that it will break the garbage collector. When a Go function calls another function, it is responsible for saving all its pointers onto the stack where the GC can see them. If you stop a function at some random instruction, there will be pointers in registers that the GC can't see. Even if you wanted to put them on the stack, you might not be able to if the compiler determined that a stack slot wasn't necessary for that particular pointer.

Disabling the GC for the duration of the function call might work, but to disable the GC you have to take locks that could be held by other threads that have been stopped by the debugger. (Though, even if you don't try to disable it, allocating memory in the called function could trigger a GC that will never finish for the same reason...)

An alternative would be to start a goroutine that calls the function. That would work from a fresh stack, so no issues with the GC. Instead there would be issues with the scheduler: there's no way to force that new goroutine to run, especially since there's no guarantee that there's a free thread. So that's not great either.

An approach that I think might actually be viable is to start a new thread (OS thread, not goroutine) that then enters the runtime as if it were a C to Go crosscall. When a non-Go thread enters Go code, a new G is allocated for it, and that G immediately runs the desired Go function. (I think that goroutine is locked to that thread for its duration, so you don't have to worry about some random goroutine using it to run, but I could be wrong). So now we'd have a new goroutine that was guaranteed to run promptly.

This could in principle be done entirely by the debugger injecting code into the debugged process, but it might be nice for the runtime to provide an actual supported hook for it.

@bcmills, @aclements, @ianlancetaylor, perhaps you have thoughts here?

[ianlancetaylor]

My main comment is that we already support calling a Go function on a new thread, since that is exactly what we do using cgo when C code starts a thread that calls a //export Go function. So in a sense we already have the hook you describe: it's called runtime.cgocallback.

[bcmills]

Presumably you want to be able to call those functions with arguments. Making the call from a different thread with local variables as arguments could easily violate the runtime's invariants about cross-goroutine access to the stack, so I think you'd have to restrict it to use only global or already-escaped variables — but that seems tricky to enforce.

[bcmills]

Another option might be to only allow function calls at GC safepoints, but then I guess we'd want some way to tell the debugger to break at the next safepoint.