Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: Go 1.14/Windows asynchronous preemption mechanism likely incompatible with debugging #36494

Open
aarzilli opened this issue Jan 10, 2020 · 3 comments

Comments

@aarzilli
Copy link
Contributor

@aarzilli aarzilli commented Jan 10, 2020

If I understand correctly asynchronous preemption is implemented in Windows by:

  1. Calling SuspendThread on the target thread
  2. Waiting for SuspendThread to take effect by calling GetThreadContext
  3. Injecting a function call on the target thread by manipulating the context and calling SetThreadContext
  4. Calling ResumeThread on the target thread

This procedure being implemented by preemptM in src/runtime/os_windows.go.

However, from what I can observe, it seems that GetThreadContext will also return when the target thread get suspended after hitting a software breakpoint (placed by a debugger). This has two effects: first the breakpoint will be missed (because preemptM will manipulate the thread context to insert a function call, masking the breakpoint), secondly the return address for the injected call will be in the middle of an instruction (because the software breakpoint was partially overwriting an instruction).

A debugger that knows about this can work around it by checking if a thread is stopped on the entry point of asyncPreempt, and fix its return address. For this to work however preemptM needs to fully finish its context manipulation, if it ends up being stopped between the point where GetThreadContext returns and SetThreadContext is called the same problem will happen.

I'm not sure that there is a way to use software breakpoints with Go 1.14 with async preemption, as it currently is. Did I misunderstand anything?

@ALTree

This comment has been minimized.

Copy link
Member

@ALTree ALTree commented Jan 10, 2020

aarzilli added a commit to aarzilli/delve that referenced this issue Jan 10, 2020
See golang/go#36494 for a description of why
full support for 1.14 under windows is problematic.
aarzilli added a commit to aarzilli/delve that referenced this issue Jan 10, 2020
See golang/go#36494 for a description of why
full support for 1.14 under windows is problematic.
@aclements

This comment has been minimized.

Copy link
Member

@aclements aclements commented Jan 10, 2020

Can you explain a bit more about how software breakpoints work under Windows (or point me to the relevant APIs)? I thought this would all be fine because SuspendThread acts like a semaphore, but I keep finding more and more little surprises with SuspendThread. :P

One possible, slightly awful work around may be for the debugging to poke a 1 into runtime.debug.asyncpreemptoff when it attaches. (Or if it's starting the process itself, add asyncpreemptoff=1 to GODEBUG.) Obviously not ideal because it may change program behavior.

@aarzilli

This comment has been minimized.

Copy link
Contributor Author

@aarzilli aarzilli commented Jan 10, 2020

Debuggers call WaitForDebugEvent, when a breakpoint happens (of any kind) the thread that executed the breakpoint is stopped and WaitForDebugEvent returns. Apparently this is also enough for any GetThreadContext call to return. A software breakpoint is just an INT 3 instruction overwriting the first byte of some other instruction, since executing it also updates the value of RIP the context returned by GetThreadContext will have RIP one byte inside whatever instruction was overwritten with the breakpoint.

One only-works-with-delve solution would be to add this between the SuspendThread call and the GetThreadContext call in preemptM:

if debuggerAttached {
    INT 3
}

with debuggerAttached being some global variable that the debugger is supposed to set after attaching. This should prevent (if I'm not wrong) preemptM from seeing the target thread's context before the debugger has a chance to and prevent preemptM from seeing the thread in a weird state.

The only other change needed in the debugger would be to ignore this breakpoint inside preemptM.

Of course this won't solve the problem for any debugger other than delve (also: I haven't tested this, so it's possible that it doesn't work at all).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.