runtime: native callback on Windows only returns 32 bit result on 64 bit platform #29331
Description
What version of Go are you using?
$ go version go version go1.11.4 windows/amd64
Problem
I'm calling a DLL function, that takes a callback function, where the callback function returns a pointer to some data structure, and the DLL will perform some action on the data structure.
package main
import (
"golang.org/x/sys/windows"
"unsafe"
)
type A struct {}
var a = &A{}
func Callback() uintptr {
ptr := uintptr(unsafe.Pointer(a))
fmt.Printf("Address: 0x%X\n", ptr)
return ptr
}
func main() {
dll := windows.MustLoadDLL("some.dll")
init := dll.MustFindProc("init")
init.Call(windows.NewCallback(Callback))
fmt.Printf("This never get prints\n")
}I'm running Go on Windows 64-bit machine, the DLL is also 64-bit. The above code will print the address of the pointer, which is 0xC000080018, but the call to the DLL will fail silently.
Cause
I don't have the source code to the DLL so I can't make it print the value it gets. So I used IDA pro to dynamically debug the program, and I found that the Go callback function is actually returning 32-bit value instead of 64-bit value. This is the final code of the Go callback routine before it returns back to the DLL:
main.exe:00000000004537E7 mov eax, [rcx+rdx-8]
main.exe:00000000004537EB pop qword ptr [rcx+rdx-8]
main.exe:00000000004537EF retn
As you can see Go is writing the return value in eax instead of rax, which will discard the higher address, therefore the DLL will only get 0x80018 and caused errors.
Possible fix?
I've done some digging into Go's implementation of the callback, and found the assembly code at src/runtime/sys_windows_amd64.s. I noticed that in runtime·callbackasm1, the return value is set using MOVL instead of MOVQ. So I changed it to MOVQ and everything worked.
I'm not sure if it's a bug or actually intended, so I'm opening an issue first.