Package all required components of the MinGW toolchain

[brson]

Easier than #9367. We're already packaging a few of MinGW's binaries. There are probably just a few more pieces needed to have a completely self contained toolchain. This will require either using the linker directly on windows or packaging all of gcc with Rust.

Nominating.

[pnkfelix]

Accepted for 1.0, assigning P-backcompat-libs (in lieu of P-backcompat-tools)

[brson]

May depend on #11937.

[brson]

My current opinion is that just bundling gcc is the path of least resistence here. gcc's dependencies are no greater than ld itself. The only sticking point is that gcc's license does include the GPL runtime exception, unlike the other components. I'm investigating the legal issues.

[thestinger]

I still think we would be better off bundling clang and using it to drive ld, because it would avoid spawning a large process. That's a significant cost on Windows slowing down every compile.

[brson]

Building clang is a possibility for the future but it is a bigger and more controversial change. Using clang to drive ld still requires bundling the GNU linker, so doesn't save us any appreciable amount of work. I would rather still just use lld directly, skipping clang entirely.

[thestinger]

The comparison would be pre-built clang (used as a library) vs. pre-built gcc. It could be built as part of building Rust but it's no more necessary than building gcc AFAICT.

[thestinger]

Well, I guess it does need to match the LLVM version.

[brson]

Still waiting on legal advice re packaging gcc.

[brson]

Packaging gcc is fine, considered 'aggregation' in GPL terms, which is allowed in most circumstances, including this one.

[brson]

Also need to package 'ar'.

[alexcrichton]

LLVM has an llvm-ar utility, and I believe that it encompasses all of our use cases, so it may not be strictly required in the long run to continue shelling out to ar, but for now I believe it's easiest to just bundle it.

[vadimcn]

Dropping these files into rust installation seems to be sufficient for self-hosting:

rust/
    bin/
        gcc.exe
        ld.exe
        ar.exe
        libgcc_s_dw2-1.dll 
        libstdc++-6.dll
    lib/
        crt2.o
        crtbegin.o
        crtend.o
        dllcrt2.o
        libadvapi32.a
        libcrypt32.a
        libgcc.a
        libgcc_eh.a
        libgcc_s.a
        libimagehlp.a
        libiphlpapi.a
        libkernel32.a
        libm.a
        libmingw32.a
        libmingwex.a
        libmsvcrt.a
        libpsapi.a
        libshell32.a
        libstdc++.a
        libuser32.a
        libws2_32.a

Note on crtbegin.o and crtend.o: a typical mingw-w64 installation contains two copies of these. In order to have working stack unwinding, you'll need the ones from lib\gcc\i686-w64-mingw32\x.y.z, not from i686-w64-mingw32\lib.

[vadimcn]

Given that gcc binaries are target-specific, it probably isn't a great idea to drop them in the root though. I'm thinking something like $(RUST)/gcc/<target triple>/bin|lib might be appropriate. Any other ideas?

[brson]

I'd like to give this some thorough testing before calling it closed.

[brson]

The Win64 installer is not working. libwinpthread-1.dll is missing (which is strange since the bots are not supposed to be using the pthread mingw). Dependency walker also says that api-ms-win-core-kernel32-private-l1-1-1.dll and two other similarly-named required DLL's aren't available; I have no idea what those are.

Win32 build tests fine!

[vadimcn]

@brson, perhaps the nightly bot got installed with "posix" mingw? We can package libwinpthread-1.dll too, but I think the preferable solution would be not having this dependency at all.

You can ignore "missing" api-ms-win-... dependencies,- they are not missing, just not on PATH, but Windows knows where to find them.

[brson]

@vadimcn I will investigate the bot configuration.

[brson]

I believe what's happening is that the distribution script is picking up the wrong libgcc_s_seh-1.dll and packaging it. msys2 comes with its own copy that does use emulated pthreads. I'm going to try switching which order the msys2/mingw-w64 paths are in in $PATH and seeing what happens.

[vadimcn]

Does that msys2 instance have gcc package installed in it? Mine doesn't, and it doesn't contain the libgcc_s_seh-1 dll either.

[retep998]

I manually removed all vestiges of msys2's mingw64 toolchain on my system to make sure I wasn't accidentally picking up any unwanted dependencies.
pacman -Rns $(pacman -Qq | grep mingw-w64-x86_64)

[brson]

@vadimcn msys2 does not appear to have gcc installed but I may have e.g. installed and uninstalled during the initial provisioning.

[brson]

The problem seems to be with the 'innoextract' tool we have installed on that bot via msys, which pulls in gcc-libs. I'll look into fixing that tomorrow.

[brson]

I've modified the win64 VM to not contain the pthread-based gcc-libs. Next nightly should work.

[brson]

New nightly still didn't work. Continuing to investigate.

[brson]

X86_64 nightlies work correctly now.

[brson]

Reopening because of complications in #17442 that make this not solved.

[brson]

Per @alexcrichton I'm going to close this because the (statically compiled) cargo binaries should work without mingw; the problem cargo is encountering is just with building.