Reconsider mingw-w64 i686 exception handling

[Martchus]

Currently, the mingw-w64-gcc Arch Linux package is configured to use SJLJ for i686. This impairs performance even when no exceptions occur. However, it works reliably.

It seems MSYS2 is using different exception handling for their i686 mingw-w64 packages. It is using DW2 for i686 (for x86_64 they also use SEH). Maybe this change for i686 would also make sense under Arch Linux?

Possible downsides

• DW2 does not allow to throw exceptions over Windows system DLLs. This might be a problem for applications which need to catch exceptions thrown inside an event handler outside of the event loop.
• Not sure about SEH.
  • I suppose it is only available under x86_64. Otherwise, why isn't MSYS2 using it for i686, too? Apparently SEH for i686 works quite differently than for x86_64. Only the x86_64 version has been implemented in GCC.
• There's not much information about DW2 support in GCC for Windows available. Does it work reliably?

Tasks

• Determine whether assertions above are actually correct.
• Update mingw-64-gcc package to enable DW2/SEH for i686
  • Use --disable-sjlj-exceptions --with-dwarf2 to enable DW2 for i686
  • or Use ??? to enable SEH for i686 not implemented in GCC
• Recompile all mingw-w64 packages because the libgcc library name would change from libgcc_s_sjlj-1.dll to libgcc_s_seh-1.dll for SEH and to libgcc_s_dw2-1.dll for DW2
  (Only the i686 version would change because x86_64 already uses SEH. Since the archs are tied that wouldn't help much, though.)

References

• https://wiki.qt.io/MinGW-64-bit
• https://github.com/Alexpux/MINGW-packages/blob/master/mingw-w64-gcc/PKGBUILD#L148
• https://gcc.gnu.org/wiki/WindowsGCCImprovements (seems outdated)

[xantares]

It's sjlj for i686 and already seh for x86_64.
Could you give a actual example of the downside of DW2 not throwing in event handlers ? Are you referring to Qt apps ?

[Martchus]

It's sjlj for i686 and already seh for x86_64.

Oh, you're right. Likely I've been looking in the wrong directory for the libgcc DLL. Then this topic actually becomes much less interesting because the most relevant architecture is using SEH already. I changed the issue description accordingly.

That also answers my question "Does it work reliably?" because apparently it does.

Could you give a actual example of the downside of DW2 not throwing in event handlers ? Are you referring to Qt apps ?

Yes, I'm referring to Qt apps for instance. But I'm not sure about this myself.
The conclusion comes from the assertion "[DW2] needs whole call stack to be dwarf-enabled, which means exceptions cannot be thrown over e.g. Windows system DLLs" in the referenced Qt Wiki article. Since Qt's callback/event-loop mechanism is built around the WinAPI this might be problematic.

[Martchus]

It might be time to switch to DWARF-2 exception handling...

(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85638#c3)

[squeevee]

Since Qt's callback/event-loop mechanism is built around the WinAPI this might be problematic.

Unhandled exceptions in event handlers are not supported by Qt, probably for this specific reason.
https://doc.qt.io/qt-5/exceptionsafety.html#exceptions-in-client-code
https://code.woboq.org/qt5/qtbase/src/corelib/kernel/qeventloop.cpp.html#194

[Martchus]

Thanks for these links. Considering that MSYS2 and Qt's official binaries are using DW2 I suppose it is save to use it as well. As far as I know @xantares already plans to do the switch when updating to GCC 10. However, it will take me a while to recompile all package of the binary repo.

[Martchus]

By the way, I'd simply increase the pkgrel of the packages by 0.1. It is unlikely that all package maintainers will increase the pkgrel of every package in a timely manner or at all. Hence it seems to be the best to handle it this way. I've already done this previously for other mass rebuilds (e.g. on a new Perl or Python release). I could also increase the pkgrel by 1 but that would mean I'd potentially miss any real packaging changes. Any thoughts regarding this?

[xantares]

Why not just wipe all mingw packages from the repo and rebuild from scratch ?

[Martchus]

I will actually start with a wiped repo to avoid any old packages being accidentally pulled into the build of a new package. But that's not actually the problem I want to tackle by increasing pkgrel. The problem are existing systems which have already several mingw-w64 packages installed. These packages need to be updated and pacman will obviously only update them if it thinks there's a newer version in the repository (which is will not when keeping the pkgrel). The official repositories increase the pkgrel as well on rebuilds, e.g. qt5-tools has updated recently.

[xantares]

hello, mingw-gcc 10.1 with dw2 seems to work fine, I plan to push the update when gcc 10.1 hits the regular repos (its in staging for now)

[Martchus]

Good to hear. I also need to create an own, separate staging repository for the rebuild, take care of the pkgrel versioning, decide which packages to get rid of (likely ANGLE and Qt 4) and handle potential build errors. I have also regular Linux and Android cross-packages in the repo which I need to take care of at the same time. So it will likely take me a while to rebuild everything and while the rebuild is ongoing I'm not going to update any packages built against the old GCC package anymore.

[xantares]

gcc 10 uses dwarf2 now

[Martchus]

I'll start the rebuild this evening or tomorrow. Seems you've already rebuilt packages locally. Anything to watch out for? (I've already read your comment under mingw-w64-glib2.)

[xantares]

Yes, I started to rebuild some packages and noticed a few issues:

• the C compiler defaults to -fno-common, so some package may fail now without patching or working around with -fcommon like mingw-w64-freeglut
• the fortran compiler is more strict with arrays so some workaround may be needed with -fallow-argument-mismatch, like mingw-w64-mumps
  I have no worries since the issues start to pop up on the forums as the Fedora folks already had to patch up their packages

[Martchus]

I've rebuilt the core toolchain and started rebuilding other packages. Unfortunately I found already failures within the first batch (1): mingw-w64-fftw, mingw-w64-freeglut, mingw-w64-fribidi, mingw-w64-lua, mingw-w64-xerces-c, mingw-w64-rust, nsis2, mingw-w64-crossc, mingw-w64-spirv-tools
I'll also the maintainers this evening and try to find fixes myself. I suppose I'll have to setup an easy way to share the build logs from my server.

(1) I'm splitting the rebuilding in batches similar to official rebuilds conducted on https://rebuilds.foutrelis.com/.

[xantares]

freeglut, nsis2, xerces, fftw should be fixed
for spirv, mingw-w64-crossc is replaced by mingw-w64-spirv-cross

[Martchus]

@xantares Thanks for taking care (I've also just seen your update for mingw-w64-lua).

In the meantime I tried my luck with mingw-w64-rust. The missing python2 dependency was of course an easy fix but the points mentioned in the pinned comment regarding the /opt/rust directory are problematic as I am using makechrootpkg and actually want to avoid such hacks. I'm nevertheless trying to workaround the problem. Annoyingly the build takes forever so I don't know yet whether my workaround works. Something seems to prevent continuing the existing build and I also don't have a ccache setup for rust yet (at least it worked for rebuilding the c/c++ bits). But if this gets to ridiculously complicated/hacky I would rather omit the package and everything depending on it (and only using mingw-w64-cairo-bootstrap or and old version of mingw-w64-librsvg).

Additionally, it seems that wine doesn't work in my chroot anymore (https://martchus.no-ip.biz/build-data/rebuild-dw2/mingw-w64-fribidi/pkg/build.log) so I have to figure out why that is the case.

[Martchus]

I finally was able to build rust, see my AUR comment regarding that package. freeglut and fftw work now as well. I'll come back to the other packages later. Luckily the 2nd batch seems already less problematic.

By the way, the problems with WINE in my systemd-nspawn container were caused because no terminal was available. I'm using this rebuild to test a new build script/system which will help me maintaining the repository in the future. It runs as systemd service and simply forks makechrootpkg. Apparently starting systemd-nspawn with --console=interactive is not enough. One also needs to set TERM via --setenv=TERM=xterm. Otherwise WINE always fails with unknown': I need something more specific. which likely comes from ncurses.

[Martchus]

I'm still having trouble with some Qt libraries (multimedia and virtualkeyboard).

The error in Qt multimedia is especially strange as it seems completely unrelated to the GCC update and I'm wondering why I didn't run into it before.

It fails in the line

i686-w64-mingw32-g++ -Wl,-O1,--sort-common,--as-needed -fstack-protector -shared -Wl,-subsystem,windows -Wl,--out-implib,/build/mingw-w64-qt5-multimedia/src/qtmultimedia-everywhere-src-5.14.2/build-i686-w64-mingw32-shared/plugins/mediaservice/libdsengine.dll.a -o ../../../plugins/mediaservice/dsengine.dll @object_script.dsengine.Release  -lgdi32 -lmf -lmfplat -lmfuuid -ld3d9 -ldxva2 -lwinmm -levr /build/mingw-w64-qt5-multimedia/src/qtmultimedia-everywhere-src-5.14.2/build-i686-w64-mingw32-shared/lib/libQt5Multimedia.dll.a /usr/i686-w64-mingw32/lib/libQt5Widgets.dll.a /usr/i686-w64-mingw32/lib/libQt5Gui.dll.a /usr/i686-w64-mingw32/lib/libQt5Network.dll.a /usr/i686-w64-mingw32/lib/libQt5Core.dll.a -lstrmiids -ldmoguids -luuid -lmsdmo -lole32 -loleaut32 -lamstrmid -lksuser -lglu32 -lopengl32 -lgdi32 -luser32

with errors like

./../i686-w64-mingw32/lib/../lib/libstrmiids.a(lib32_libstrmiids_a-strmiids.o):strmiids.c:(.rdata$IID_IAMResourceControl[_IID_IAMResourceControl]+0x0): first defined here
/usr/lib/gcc/i686-w64-mingw32/10.1.0/../../../../i686-w64-mingw32/bin/ld: /usr/lib/gcc/i686-w64-mingw32/10.1.0/../../../../i686-w64-mingw32/lib/../lib/libamstrmid.a(lib32_libamstrmid_a-amstrmid.o):amstrmid.c:(.rdata+0x380): multiple definition of `IID_IAMStreamSelect'; /usr/lib/gcc/i686-w64-mingw32/10.1.0/../../../../i686-w64-mingw32/lib/../lib/libstrmiids.a(lib32_libstrmiids_a-strmiids.o):strmiids.c:(.rdata$IID_IAMStreamSelect[_IID_IAMStreamSelect]+0x0): first defined here

so -lstrmiids and -lamstrmid contain conflicting symbols. The annoying problem (which I couldn't solve so far) is that leaving out one of those libraries produces a different set of undefined reference to errors. So both libs are needed but also conflicting with each other.

Maybe this is a mingw-w64 bug? Unfortunately I couldn't find much information about these libraries. Likely I'll just workaround with -Wl,--allow-multiple-definition for now.

[xantares]

that's definetely related to gcc 10. I guess this is a problem in crt.

[Martchus]

I've seen you've amended your comment. Neverthless I've tried the -fcommon flag at least on the linker line and it doesn't make a difference. Because the linker errors are only between symbols of libraries provided by the CRT I guess it is a CRT bug as well.

[Martchus]

I've rebuilt everything now. I dropped the following packages in the process due to build errrors:

mingw-w64-wxmsw
mingw-w64-enca
mingw-w64-libxslt
mingw-w64-gtk2
nsis
nsis2 # didn't try to rebuild after xantares fixed it because I don't need it

I have also removed some other package even before the rebuild because they are outdated and apparently not actively maintained (e.g. ANGLE, KDE libs, FileZilla, Geany, Evince).

I can re-add removed packages when they are updated or fixed (or wanted in case of nsis2).

I've also just briefly tested whether the resulting executables actually work natively under Windows. I've tested Qt apps, x264, x265, ffmpeg and PostgreSQL apps. It seems to be the case for both architectures. Statically linked Qt apps work as well. In particular, also the exception handling seems to work (the tested apps follow what @squeevee mentioned like every sane app should).

So I'm going to move the packages from the staging repository to my regular repository tomorrow.

---

By the way, I worked around the issue in qt5-multimedia via -Wl,--allow-multiple-definition. People on the mingw-w64 IRC channel say it is safe. The conflicting symbols are not necassarily wrong but are actually supposed to be ignored via __declspec(selectany) (see 030f60a). Not sure why it doesn't work but apparently the Git version of the CRT no longer has this problem so I just use that workaround for now.

[xantares]

actually I pushed a fix for nsis, doesnt it build now ?

[Martchus]

I could not build nsis. I haven't built nsis2 yet because the download servers seem currently down.

Since all packages have been moved from the staging repository to the actual repository I'm closing the issue.