cmd/link: ld consumes infinite memory when linking cgo app

[Elemecca]

What version of Go are you using (go version)?

$ go version
go version go1.21.1 windows/amd64

Does this issue reproduce with the latest release?

1.21.1 is the latest version available in MSYS2. I tried the official Windows binary release of 1.21.2, but it doesn't support cgo so it fails before the linking step. For what it's worth I don't see anything that looks relevant in the commits between 1.21.1 and 1.21.2.

What operating system and processor architecture are you using (go env)?

Windows 11, amd64
go from the MSYS2 package mingw-w64-ucrt-x86_64-go
GNU Binutils 2.41 from the MSYS2 package mingw-w64-ucrt-x86_64-binutils

go env Output

$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\maia\AppData\Local\go-build
set GOENV=C:\Users\maia\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\maia\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\maia\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:/msys64/ucrt64/lib/go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLCHAIN=auto
set GOTOOLDIR=C:\msys64\ucrt64\lib\go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.21.1
set GCCGO=gccgo
set GOAMD64=v1
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\maia\code\snafu\go-hotplug\go.mod
set GOWORK=
set CGO_CFLAGS=-O2 -g
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-O2 -g
set CGO_FFLAGS=-O2 -g
set CGO_LDFLAGS=-O2 -g
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\msys64\tmp\go-build1188127562=/tmp/go-build -gno-record-gcc-switches

What did you do?

Attempt to build the example program in elemecca/go-hotplug

go build -x ./examples/announcehid

What did you expect to see?

It should successfully produce an executable, or promptly fail if I did something wrong in my code. Either way it should consume a reasonable amount of memory while doing so.

What did you see instead?

During the cmd/link step at the end of the build, ld.exe allocates memory continuously and never finishes.

On my machine, which has 64 GB of RAM, it allocates about 2 GB/second until it hits somewhere around 60 GB, then slows down when the system starts swapping furiously. For science I let it keep running once; Windows bluescreened when it had allocated about 215 GB.

Since the issue is in ld.exe it's possible this is an upstream bug in GNU Binutils, but I don't know enough about how cmd/link works to determine that or to give them enough information for a bug report.

[Elemecca]

I tried building go from source in my MSYS2 environment. Both go1.21.2 and master at f711892a8a exhibit the same behavior as the packaged go 1.21.1.

Also, for posterity the current commit in elemecca/go-hotplug at the time of this issue is 9e9ddbf41d.

[Elemecca]

I managed to reduce the test case down to just this.

package main

import (
    "fmt"
    "unsafe"
)

/*
    #cgo LDFLAGS: -lcfgmgr32
    #define WINVER 0x0602 // Windows 8
    #define UNICODE
    #include <windows.h>
    #include <cfgmgr32.h>
    #include <devpkey.h>

    // this is missing from cfgmgr32.h in mingw-w64
    CMAPI CONFIGRET CM_Get_Device_Interface_PropertyW(LPCWSTR pszDeviceInterface, const DEVPROPKEY *PropertyKey, DEVPROPTYPE *PropertyType, PBYTE PropertyBuffer, PULONG PropertyBufferSize, ULONG ulFlags);
*/
import "C"

func main() {
    var symbolicLink *C.WCHAR
    var propType C.DEVPROPTYPE
    var size C.ULONG
    var devInstanceId [C.MAX_DEVICE_ID_LEN + 1]C.WCHAR
    var deviceInstance C.DEVINST

    size = (C.ULONG)(len(devInstanceId) * C.sizeof_WCHAR)
    C.CM_Get_Device_Interface_PropertyW(
        symbolicLink,
        &C.DEVPKEY_Device_InstanceId,
        &propType,
        (C.PBYTE)(unsafe.Pointer(&devInstanceId[0])),
        &size,
        0,
    )

    C.CM_Locate_DevNodeW(
        &deviceInstance,
        &devInstanceId[0],
        C.CM_LOCATE_DEVNODE_NORMAL,
    )

    size = 4
    var address int32
    C.CM_Get_DevNode_PropertyW(
        deviceInstance,
        &C.DEVPKEY_Device_Address,
        &propType,
        (C.PBYTE)(unsafe.Pointer(&address)),
        &size,
        0,
    )

    fmt.Print(address)
}

[dmitshur]

CC @golang/compiler.

[mknyszek]

In triage, we're wondering if you were to write an equivalent C program (with the linker invoked in a similar way (i.e. similar flags), will ld.exe still consume a lot of memory? If so, that seems like a bug in ld.exe.

[Elemecca]

@mknyszek that's worth a shot.

It seems that go build is calling link.exe with the command line

C:\msys64\ucrt64\lib\go\pkg\tool\windows_amd64\link.exe -o C:\msys64\tmp\go-build323995550\b001\exe\a.out.exe -importcfg C:\msys64\tmp\go-build323995550\b001\importcfg.link -buildmode=pie -buildid=hmtQRCYZr6LXF52m_PMN/lIjeZR2XoBH9yVTDiytc/ZhhCE-YcTTorCMxaZknw/hmtQRCYZr6LXF52m_PMN -extld=gcc C:\msys64\tmp\go-build323995550\b001\_pkg_.a

link.exe is in turn calling gcc.exe with the command line

gcc -m64 -mconsole -Wl,--tsaware -Wl,--nxcompat -Wl,--major-os-version=6 -Wl,--minor-os-version=1 -Wl,--major-subsystem-version=6 -Wl,--minor-subsystem-version=1 -Wl,--dynamicbase -Wl,--high-entropy-va -o C:\msys64\tmp\go-build323995550\b001\exe\a.out.exe -Wl,--no-insert-timestamp C:\msys64\tmp\go-link-201239192\go.o C:\msys64\tmp\go-link-201239192\000000.o C:\msys64\tmp\go-link-201239192\000001.o C:\msys64\tmp\go-link-201239192\000002.o C:\msys64\tmp\go-link-201239192\000003.o C:\msys64\tmp\go-link-201239192\000004.o C:\msys64\tmp\go-link-201239192\000005.o C:\msys64\tmp\go-link-201239192\000006.o C:\msys64\tmp\go-link-201239192\000007.o C:\msys64\tmp\go-link-201239192\000008.o C:\msys64\tmp\go-link-201239192\000009.o -O2 -g -lcfgmgr32 -O2 -g -Wl,-T,C:\msys64\tmp\go-link-201239192\fix_debug_gdb_scripts.ld -Wl,--start-group -lmingwex -lmingw32 -Wl,--end-group -lkernel32

gcc.exe calls collect2.exe with the command line

C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/collect2.exe -plugin C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/liblto_plugin.dll -plugin-opt=C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/lto-wrapper.exe -plugin-opt=-fresolution=C:\msys64\tmp\cc6t6Uh9.res -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-ladvapi32 -plugin-opt=-pass-through=-lshell32 -plugin-opt=-pass-through=-luser32 -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -m i386pep --subsystem console -Bdynamic -o C:\msys64\tmp\go-build323995550\b001\exe\a.out.exe C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../lib/crt2.o C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/crtbegin.o -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0 -LC:/msys64/ucrt64/bin/../lib/gcc -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/lib/../lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../.. --tsaware --nxcompat --major-os-version=6 --minor-os-version=1 --major-subsystem-version=6 --minor-subsystem-version=1 --dynamicbase --high-entropy-va --no-insert-timestamp C:\msys64\tmp\go-link-201239192\go.o C:\msys64\tmp\go-link-201239192\000000.o C:\msys64\tmp\go-link-201239192\000001.o C:\msys64\tmp\go-link-201239192\000002.o C:\msys64\tmp\go-link-201239192\000003.o C:\msys64\tmp\go-link-201239192\000004.o C:\msys64\tmp\go-link-201239192\000005.o C:\msys64\tmp\go-link-201239192\000006.o C:\msys64\tmp\go-link-201239192\000007.o C:\msys64\tmp\go-link-201239192\000008.o C:\msys64\tmp\go-link-201239192\000009.o -lcfgmgr32 -T C:\msys64\tmp\go-link-201239192\fix_debug_gdb_scripts.ld --start-group -lmingwex -lmingw32 --end-group -lkernel32 -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt -lkernel32 -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt -lkernel32 C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../lib/default-manifest.o C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/crtend.o

and finally collect2.exe calls ld.exe with the command line

C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe -plugin C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/liblto_plugin.dll -plugin-opt=C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/lto-wrapper.exe -plugin-opt=-fresolution=C:\msys64\tmp\cc6t6Uh9.res -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-ladvapi32 -plugin-opt=-pass-through=-lshell32 -plugin-opt=-pass-through=-luser32 -plugin-opt=-pass-through=-lkernel32 -plugin-opt=-pass-through=-lmingw32 -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_eh -plugin-opt=-pass-through=-lmoldname -plugin-opt=-pass-through=-lmingwex -plugin-opt=-pass-through=-lmsvcrt -plugin-opt=-pass-through=-lkernel32 -m i386pep --subsystem console -Bdynamic -o C:\msys64\tmp\go-build323995550\b001\exe\a.out.exe C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../lib/crt2.o C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/crtbegin.o -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0 -LC:/msys64/ucrt64/bin/../lib/gcc -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/lib/../lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/lib -LC:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../.. --tsaware --nxcompat --major-os-version=6 --minor-os-version=1 --major-subsystem-version=6 --minor-subsystem-version=1 --dynamicbase --high-entropy-va --no-insert-timestamp C:\msys64\tmp\go-link-201239192\go.o C:\msys64\tmp\go-link-201239192\000000.o C:\msys64\tmp\go-link-201239192\000001.o C:\msys64\tmp\go-link-201239192\000002.o C:\msys64\tmp\go-link-201239192\000003.o C:\msys64\tmp\go-link-201239192\000004.o C:\msys64\tmp\go-link-201239192\000005.o C:\msys64\tmp\go-link-201239192\000006.o C:\msys64\tmp\go-link-201239192\000007.o C:\msys64\tmp\go-link-201239192\000008.o C:\msys64\tmp\go-link-201239192\000009.o -lcfgmgr32 -T C:\msys64\tmp\go-link-201239192\fix_debug_gdb_scripts.ld --start-group -lmingwex -lmingw32 --end-group -lkernel32 -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt -lkernel32 -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc -lgcc_eh -lmoldname -lmingwex -lmsvcrt -lkernel32 C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../lib/default-manifest.o C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/crtend.o

The equivalent C program for my go test case is

#define WINVER 0x0602 // Windows 8
#define UNICODE
#include <windows.h>
#include <cfgmgr32.h>
#include <devpkey.h>
#include <stdio.h>

// this is missing from cfgmgr32.h in mingw-w64
CMAPI CONFIGRET CM_Get_Device_Interface_PropertyW(LPCWSTR pszDeviceInterface, const DEVPROPKEY *PropertyKey, DEVPROPTYPE *PropertyType, PBYTE PropertyBuffer, PULONG PropertyBufferSize, ULONG ulFlags);

int main() {
    WCHAR *symbolicLink;
    DEVPROPTYPE propType;
    ULONG size;
    WCHAR devInstanceId[MAX_DEVICE_ID_LEN + 1];
    DEVINST deviceInstance;
    LONG address;

    size = sizeof(devInstanceId);
    CM_Get_Device_Interface_PropertyW(
        symbolicLink,
        &DEVPKEY_Device_InstanceId,
        &propType,
        (PBYTE)devInstanceId,
        &size,
        0
    );

    CM_Locate_DevNodeW(
        &deviceInstance,
        devInstanceId,
        CM_LOCATE_DEVNODE_NORMAL
    );

    size = sizeof(address);
    CM_Get_DevNode_PropertyW(
        deviceInstance,
        &DEVPKEY_Device_Address,
        &propType,
        (PBYTE)&address,
        &size,
        0
    );

    printf("%d\n", address);
}

Compiling that to an object file with gcc succeeds.

gcc -c -o main.o main.c

I tried a linking command roughly equivalent to what go is using:

gcc -m64 -mconsole -Wl,--tsaware -Wl,--nxcompat -Wl,--major-os-version=6 -Wl,--minor-os-version=1 -Wl,--major-subsystem-version=6 -Wl,--minor-subsystem-version=1 -Wl,--dynamicbase -Wl,--high-entropy-va -o main.exe -Wl,--no-insert-timestamp main.o -O2 -g -lcfgmgr32 -O2 -g  -Wl,--start-group -lmingwex -lmingw32 -Wl,--end-group -lkernel32

That quickly fails with this output:

C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: main.o:main.c:(.rdata$.refptr.DEVPKEY_Device_Address[.refptr.DEVPKEY_Device_Address]+0x0): undefined reference to `DEVPKEY_Device_Address'
C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64-mingw32/13.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: main.o:main.c:(.rdata$.refptr.DEVPKEY_Device_InstanceId[.refptr.DEVPKEY_Device_InstanceId]+0x0): undefined reference to `DEVPKEY_Device_InstanceId'
collect2.exe: error: ld returned 1 exit status

Which is the same thing that I got from go build when I was trying to reduce the test case and removed something necessary to replicate the memory leak, so I suspect the go build would also fail with those errors if it wasn't hitting the runaway memory leak.

The only things I changed from the gcc command that cmd/link is using to get the one I tested with were removing the linker script -Wl,-T,C:\msys64\tmp\go-link-201239192\fix_debug_gdb_scripts.ld and replacing the list of object files.

Given that, I think whatever is triggering the issue is either in whatever extra C code cgo is adding to its output, in an object file generated by go, or in the linker script cmd/link is using. Of course, that doesn't mean it isn't an upstream bug in GNU ld, just that some of what go is doing is necessary to replicate it.

[Elemecca]

For what it's worth I also tried the MSYS2 MINGW64 environment (packages mingw-w64-x86_64-go and mingw-w64-x86_64-binutils) which uses MSVCRT instead of UCRT. Unfortunately it has the same issue.

[Elemecca]

I isolated the trigger of the issue to just the go.o file produced by go and included in the final link command that has the runaway memory leak. I can reproduce the issue with just gcc go.o.

I can't attach object files to this issue, so I put it in a gist:
https://gist.github.com/Elemecca/377451208c45b80226a39456e69f2eef