cmd/link: ld consumes infinite memory when linking cgo app #63437
Comments
|
I tried building go from source in my MSYS2 environment. Both Also, for posterity the current commit in elemecca/go-hotplug at the time of this issue is |
|
I managed to reduce the test case down to just this. package main
import (
"fmt"
"unsafe"
)
/*
#cgo LDFLAGS: -lcfgmgr32
#define WINVER 0x0602 // Windows 8
#define UNICODE
#include <windows.h>
#include <cfgmgr32.h>
#include <devpkey.h>
// this is missing from cfgmgr32.h in mingw-w64
CMAPI CONFIGRET CM_Get_Device_Interface_PropertyW(LPCWSTR pszDeviceInterface, const DEVPROPKEY *PropertyKey, DEVPROPTYPE *PropertyType, PBYTE PropertyBuffer, PULONG PropertyBufferSize, ULONG ulFlags);
*/
import "C"
func main() {
var symbolicLink *C.WCHAR
var propType C.DEVPROPTYPE
var size C.ULONG
var devInstanceId [C.MAX_DEVICE_ID_LEN + 1]C.WCHAR
var deviceInstance C.DEVINST
size = (C.ULONG)(len(devInstanceId) * C.sizeof_WCHAR)
C.CM_Get_Device_Interface_PropertyW(
symbolicLink,
&C.DEVPKEY_Device_InstanceId,
&propType,
(C.PBYTE)(unsafe.Pointer(&devInstanceId[0])),
&size,
0,
)
C.CM_Locate_DevNodeW(
&deviceInstance,
&devInstanceId[0],
C.CM_LOCATE_DEVNODE_NORMAL,
)
size = 4
var address int32
C.CM_Get_DevNode_PropertyW(
deviceInstance,
&C.DEVPKEY_Device_Address,
&propType,
(C.PBYTE)(unsafe.Pointer(&address)),
&size,
0,
)
fmt.Print(address)
} |
dmitshur
added
the
NeedsInvestigation
|
CC @golang/compiler. |
gopherbot
added
the
compiler/runtime
|
In triage, we're wondering if you were to write an equivalent C program (with the linker invoked in a similar way (i.e. similar flags), will |
mknyszek
added
the
WaitingForInfo
|
@mknyszek that's worth a shot. It seems that
and finally The equivalent C program for my go test case is #define WINVER 0x0602 // Windows 8
#define UNICODE
#include <windows.h>
#include <cfgmgr32.h>
#include <devpkey.h>
#include <stdio.h>
// this is missing from cfgmgr32.h in mingw-w64
CMAPI CONFIGRET CM_Get_Device_Interface_PropertyW(LPCWSTR pszDeviceInterface, const DEVPROPKEY *PropertyKey, DEVPROPTYPE *PropertyType, PBYTE PropertyBuffer, PULONG PropertyBufferSize, ULONG ulFlags);
int main() {
WCHAR *symbolicLink;
DEVPROPTYPE propType;
ULONG size;
WCHAR devInstanceId[MAX_DEVICE_ID_LEN + 1];
DEVINST deviceInstance;
LONG address;
size = sizeof(devInstanceId);
CM_Get_Device_Interface_PropertyW(
symbolicLink,
&DEVPKEY_Device_InstanceId,
&propType,
(PBYTE)devInstanceId,
&size,
0
);
CM_Locate_DevNodeW(
&deviceInstance,
devInstanceId,
CM_LOCATE_DEVNODE_NORMAL
);
size = sizeof(address);
CM_Get_DevNode_PropertyW(
deviceInstance,
&DEVPKEY_Device_Address,
&propType,
(PBYTE)&address,
&size,
0
);
printf("%d\n", address);
}Compiling that to an object file with I tried a linking command roughly equivalent to what go is using: That quickly fails with this output: Which is the same thing that I got from The only things I changed from the Given that, I think whatever is triggering the issue is either in whatever extra C code cgo is adding to its output, in an object file generated by go, or in the linker script |
|
For what it's worth I also tried the MSYS2 MINGW64 environment (packages |
|
I isolated the trigger of the issue to just the I can't attach object files to this issue, so I put it in a gist: |
|
It appears that the undefined references to I figured out how to fix those, which is to add an include of Adding that to the go test case fixes the issue - it now links successfully in the same environment where it used to cause the runaway memory leak. The same is true of my actual application. Updated go test case which does not trigger the runaway memory leakpackage main
import (
"fmt"
"unsafe"
)
/*
#cgo LDFLAGS: -lcfgmgr32
#define WINVER 0x0602 // Windows 8
#define UNICODE
#include <windows.h>
#include <cfgmgr32.h>
#include <initguid.h>
#include <devpkey.h>
// this is missing from cfgmgr32.h in mingw-w64
CMAPI CONFIGRET CM_Get_Device_Interface_PropertyW(LPCWSTR pszDeviceInterface, const DEVPROPKEY *PropertyKey, DEVPROPTYPE *PropertyType, PBYTE PropertyBuffer, PULONG PropertyBufferSize, ULONG ulFlags);
*/
import "C"
func main() {
var symbolicLink *C.WCHAR
var propType C.DEVPROPTYPE
var size C.ULONG
var devInstanceId [C.MAX_DEVICE_ID_LEN + 1]C.WCHAR
var deviceInstance C.DEVINST
size = (C.ULONG)(len(devInstanceId) * C.sizeof_WCHAR)
C.CM_Get_Device_Interface_PropertyW(
symbolicLink,
&C.DEVPKEY_Device_InstanceId,
&propType,
(C.PBYTE)(unsafe.Pointer(&devInstanceId[0])),
&size,
0,
)
C.CM_Locate_DevNodeW(
&deviceInstance,
&devInstanceId[0],
C.CM_LOCATE_DEVNODE_NORMAL,
)
size = 4
var address int32
C.CM_Get_DevNode_PropertyW(
deviceInstance,
&C.DEVPKEY_Device_Address,
&propType,
(C.PBYTE)(unsafe.Pointer(&address)),
&size,
0,
)
fmt.Print(address)
}However, the issue is still an issue: even in the presence of invalid input the linker should not bluescreen the computer by eating all of its RAM. |
seankhliao
removed
the
WaitingForInfo
What version of Go are you using (
go version)?Does this issue reproduce with the latest release?
1.21.1 is the latest version available in MSYS2. I tried the official Windows binary release of 1.21.2, but it doesn't support cgo so it fails before the linking step. For what it's worth I don't see anything that looks relevant in the commits between 1.21.1 and 1.21.2.
What operating system and processor architecture are you using (
go env)?Windows 11, amd64
go from the MSYS2 package
mingw-w64-ucrt-x86_64-goGNU Binutils 2.41 from the MSYS2 package
mingw-w64-ucrt-x86_64-binutilsgo envOutputWhat did you do?
Attempt to build the example program in elemecca/go-hotplug
What did you expect to see?
It should successfully produce an executable, or promptly fail if I did something wrong in my code. Either way it should consume a reasonable amount of memory while doing so.
What did you see instead?
During the
cmd/linkstep at the end of the build,ld.exeallocates memory continuously and never finishes.On my machine, which has 64 GB of RAM, it allocates about 2 GB/second until it hits somewhere around 60 GB, then slows down when the system starts swapping furiously. For science I let it keep running once; Windows bluescreened when it had allocated about 215 GB.
Since the issue is in
ld.exeit's possible this is an upstream bug in GNU Binutils, but I don't know enough about howcmd/linkworks to determine that or to give them enough information for a bug report.The text was updated successfully, but these errors were encountered: