Compiling projects with GCC 12.1 UCRT is painfully slow

[Panjaksli]

(Latest build)
MSVCRT works as expected.

[brechtsanders]

Do you have this on more than one system?

Do you know how to reproduce this so I can test for myself?

Is the problem the same when antivirus is disabled? (Which antivirus do you use?)

[Panjaksli]

I only have laptop, so there is that :D

I mostly noticed this with cmake when compiling targets with many (5+) header files containing classes/inline functions.

MS defender, works the same with it disabled.

Also by painfully slow, I mean 20s with UCRT instead of 5 with MSVCRT.

[brechtsanders]

I can't reproduce your issue.
Are there any other differences between your UCRT and MSVCRT setup?
Are you using command line (Command Prompt, PowerShell, MSYS), an IDE, or some kind of build system (make, cmake, ninja, ...)?

[Phantom-Trigger]

Hi, maybe not the accurate information, but I have the same bad compiling performance.
What's different is that both my UCRT and MSVCRT compile slowsy compared with gcc11.2.0. It's possible that this isn't a bug but the gcc's feature? The problem might be related to the headers.

I'm using the VSCode extension Code Runner, as far as I know, it simply runs the two command lines. For example:
compiled with gcc12.2.0

compiled with gcc11.2.0

And it's more painful when using bits/stdc++.h
compiled with gcc12.2.0

compiled with gcc11.2.0

(My computer is a bit out of date and slow...)

As is seen, 12.2.0 compiles approximately 2x slower then 11.2.0, and when tested on Command Prompt and PowerShell, the result is the same.

Could someone tell me why the compiling is so slow? Thanks!

[brechtsanders]

Since GCC 12 there is issue https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105858 which breaks precompiled headers and which for now requires GCC12 to be built without precompiled headers.

This is likely to have some performance impact. Maybe this is what you are experiencing?

[Phantom-Trigger]

Ah, thanks for replying! Maybe that's the reason.

[xtemp09]

I noticed that the GCC developers have no interest in fixing bugs on Windows platform. They don't even consider patches sent. clang/LLVM developers show fundamental difference.

Perhaps, migration to clang/LLVM will relieve one's migraine.

[brechtsanders]

I too have logged issues with GCC that barely got any response. Though some of them were resolved in the end.

LLVM/CLang is a different track, but it's not always an exact drop-in replacement for GCC.

But at least with LLVM/CLang Windows on ARM is supported, while GCC doesn't even look like it's working on that either.

[rockcavera]

I use C compiler as a means to compile the C code generated by Nim. Since the 12.x.x releases of gcc, I have noticed a huge slowdown compared to the 11.x.x version. With gcc 11.3.0 Nim bootstrap took 3min05s. Already with gcc 12.2.0 it is taking 8min24s. I'm using msvcrt.

[brechtsanders]

For speed testing purposes I have just made this release (it reverts binutils back to the previous version):
https://github.com/brechtsanders/winlibs_mingw/releases/tag/12.2.1-snapshot20221126-10.0.0-msvcrt-r1

Can you try if this version works faster for you?

[sisyphus]

Can you try if this version works faster for you?

I reported a similar issue in #114 - which I subsequently closed because I'm not sure that it's a bug report that should have been filed on this forum.

As regards that slowness, I can report that this snapshot makes no appreciable difference at all.

However, @brechtsanders, I do thank you for going to the trouble of providing this snapshot.
I take it that, as regards my issue with 12.2.0, I can now rule out binutils as being the culprit.

Cheers,
Rob

[rockcavera]

For speed testing purposes I have just made this release (it reverts binutils back to the previous version): https://github.com/brechtsanders/winlibs_mingw/releases/tag/12.2.1-snapshot20221126-10.0.0-msvcrt-r1

Can you try if this version works faster for you?

I tested and got this: 8min44s

[brechtsanders]

I was able to build the version with precompiled headers enabled. Can you compare speed again?
https://github.com/brechtsanders/winlibs_mingw/releases/tag/12.2.1-snapshot20221126-10.0.0-msvcrt-r2

[rockcavera]

I was able to build the version with precompiled headers enabled. Can you compare speed again? https://github.com/brechtsanders/winlibs_mingw/releases/tag/12.2.1-snapshot20221126-10.0.0-msvcrt-r2

This new release took 8min29s

I have a table here comparing build times on Nim bootstrap, all using the same conditions. Perhaps by looking at your best performing builds you can determine what else it might be.

Compiler                                                          Build Time
winlibs-x86_64-posix-seh-gcc-12.2.0-mingw-w64msvcrt-10.0.0-r2     00:08:24
winlibs-x86_64-posix-seh-gcc-11.3.0-mingw-w64msvcrt-10.0.0-r3     00:03:05
winlibs-x86_64-posix-seh-gcc-10.4.0-mingw-w64msvcrt-10.0.0-r1     00:07:39
winlibs-x86_64-posix-seh-gcc-9.5.0-mingw-w64msvcrt-10.0.0-r1      00:05:19
winlibs-x86_64-posix-seh-gcc-8.5.0-mingw-w64-9.0.0-r1             00:03:02
winlibs-x86_64-posix-seh-gcc-7.5.0-mingw-w64-7.0.0-r1             00:02:34
mingw-builds-x86_64-6.4.0-release-posix-seh-rt_v5-rev0            00:02:31
mingw-builds-x86_64-5.4.0-release-posix-seh-rt_v5-rev0            00:02:27
mingw-builds-x86_64-4.9.4-release-posix-seh-rt_v5-rev0            00:02:32
mingw-builds-x86_64-4.8.1-release-posix-seh-rt_v3-rev2            00:02:31
tdm64-gcc-10.3.0-2                                                00:03:24
clang-llvm-mingw-20220906-msvcrt-x86_64                           00:02:15
clang-15.0.5-posix-x86_64-pc-windows-msvc                         00:02:12

[rockcavera]

I tested this gcc 12.2.0 binary (x86_64-12.2.0-release-posix-seh-rt_v10-rev1) and got 3min03s under the same conditions I did the previous tests.

[sisyphus]

That build by niXman of 12.2.0 (x86_64-12.2.0-release-posix-seh-rt_v10-rev1) also eradicates the slowness I reported in #114.
@brechtsanders, are you able to pinpoint the difference that accounts for this ?

Cheers,
Rob

[brechtsanders]

A good next step would be to compare outputs of gcc -v to find out where the difference lies.

[rockcavera]

A good next step would be to compare outputs of gcc -v to find out where the difference lies.

I took a look and there are several differences, but for those who do not compile gcc, it is difficult to know what could be causing this slowdown in your release.

Winlibs configuration that differs from niXman, either because it does not have a corresponding one or because it is disabled or enabled.:

--disable-bootstrap
--disable-default-ssp
--disable-gold
--disable-libunwind-exceptions
--disable-serial-configure
--disable-sjlj-exceptions
--disable-stage1-checking
--disable-version-specific-runtime-libs
--enable-__cxa_atexit
--enable-clocale=generic
--enable-install-libiberty
--enable-languages=c,c++,fortran,lto,objc,obj-c++,jit
--enable-ld
--enable-libada
--enable-libquadmath
--enable-libssp
--enable-libstdcxx
--enable-libstdcxx-backtrace
--enable-mingw-wildcard
--enable-offload-targets=nvptx-none
--enable-plugin
--with-build-sysroot=/r/winlibs64_stage/17520.gcc/gcc-12.2.0/build_mingw/mingw-w64 CFLAGS='-I/d/Prog/winlibs64_stage/custombuilt/include/libdl-win32 -D__USE_MINGW_ACCESS' CXXFLAGS=-D__USE_MINGW_ACCESS LDFLAGS='-Wl,--disable-nxcompat -Wl,--disable-high-entropy-va -Wl,--disable-dynamicbase'
--with-diagnostics-color=auto
--with-gmp=/d/Prog/winlibs64_stage/custombuilt
--with-isl=/d/Prog/winlibs64_stage/custombuilt
--with-mpc=/d/Prog/winlibs64_stage/custombuilt
--with-mpfr=/d/Prog/winlibs64_stage/custombuilt
--with-stabs
--with-tune=generic
--without-included-gettext

niXman settings that differs from winlibs, either because it does not have a corresponding one or because it is disabled or enabled:

--disable-werror
--enable-bootstrap
--enable-languages=c,c++,fortran,lto
--enable-libatomic
--enable-libstdcxx-filesystem-ts=yes
--enable-version-specific-runtime-libs
--target=x86_64-w64-mingw32
--with-arch=nocona
--with-boot-ldflags=' -Wl,--disable-dynamicbase -static-libstdc++ -static-libgcc'
--with-bugurl=https://sourceforge.net/projects/mingw-w64
CFLAGS='-O2 -pipe -fno-ident -I/c/buildroot/x86_64-1220-posix-seh-rt_v10-rev1/mingw64/opt/include -I/c/buildroot/prerequisites/x86_64-zlib-static/include -I/c/buildroot/prerequisites/x86_64-w64-mingw32-static/include'
CXXFLAGS='-O2 -pipe -fno-ident -I/c/buildroot/x86_64-1220-posix-seh-rt_v10-rev1/mingw64/opt/include -I/c/buildroot/prerequisites/x86_64-zlib-static/include -I/c/buildroot/prerequisites/x86_64-w64-mingw32-static/include'
CPPFLAGS=' -I/c/buildroot/x86_64-1220-posix-seh-rt_v10-rev1/mingw64/opt/include -I/c/buildroot/prerequisites/x86_64-zlib-static/include -I/c/buildroot/prerequisites/x86_64-w64-mingw32-static/include'
LDFLAGS='-pipe -fno-ident -L/c/buildroot/x86_64-1220-posix-seh-rt_v10-rev1/mingw64/opt/lib -L/c/buildroot/prerequisites/x86_64-zlib-static/lib -L/c/buildroot/prerequisites/x86_64-w64-mingw32-static/lib '
LD_FOR_TARGET=/c/buildroot/x86_64-1220-posix-seh-rt_v10-rev1/mingw64/bin/ld.exe
--with-gmp=/c/buildroot/prerequisites/x86_64-w64-mingw32-static
--with-gnu-as
--with-gnu-ld
--with-isl=/c/buildroot/prerequisites/x86_64-w64-mingw32-static
--with-mpc=/c/buildroot/prerequisites/x86_64-w64-mingw32-static
--with-mpfr=/c/buildroot/prerequisites/x86_64-w64-mingw32-static
--with-sysroot=/c/buildroot/x86_64-1220-posix-seh-rt_v10-rev1/mingw64
--with-tune=core2

[brechtsanders]

I have tried to patch the PCH issue according to MSYS2's patches.
Can you please check if the new release fixes any PCH and/or C++ compilation speed issues?

[rockcavera]

@brechtsanders unfortunately it didn't solve the speed problem.

[term-est]

12.2 feels a lot faster, and it seems like PCH is working too

Thank you for the release 🍓

[brechtsanders]

@term-est Are you specifically referring to yesterday's build?

I also had the impression it was working smoother.
I have a feeling things got slower since this issue, and yesterday's build finally had it patched.

But that's just a gut feeling, maybe other things were at play too.

[rockcavera]

@term-est Are you specifically referring to yesterday's build?

I downloaded and tested release 4, but the compile time remains the same. That is, it remains 2.5x slower than the one distributed by niXman.

[rockcavera]

@brechtsanders I recently tested gcc version 13.2.0 (MinGW-W64 x86_64-msvcrt-posix-seh, built by Brecht Sanders, r4) and it seems that the slowness problem has been resolved. I verified that you now use the -march=nocona switch. Was that it?

[brechtsanders]

I'm not sure what caused it.
I keep improving the build process with each version, but also each version uses the latest version of all dependencies, so the issue may have been resolved in one of those.
I'm glad it's resolved now.