Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Godot 4 compiled with MinGW is crashing with a white screen [GCC 11.x bug fixed in 11.3] #51070

Closed
MmAaXx500 opened this issue Jul 30, 2021 · 28 comments

Comments

@MmAaXx500
Copy link
Contributor

MmAaXx500 commented Jul 30, 2021

Godot version

4.0 custom build (bdcc874)

System information

Windows 10 21H1

Issue description

The editor is crashing with a white screen on startup, with or without a project. I have been tried some combinations of compilers and options:
MSVC with debug -> OK
MSVC with release_debug -> OK
MinGW with debug -> OK
MinGW with release_debug -> Crash

This issue has been introduced with the 99fe462 commit.

Output of drmingw
godot.windows.opt.tools.64.exe caused an Access Violation at location 00007FF73A4C0000 DEP violation at location 00007FF73A4C0000.

AddrPC           Params
00007FF73A4C0000 00000215616FA200 0000003CE61FF5A8 0000003CE61FF5B0
00007FF77C559842 0000000000400000 00007FF77CA33F03 0000000000400000  godot.windows.opt.tools.64.exe!RenderingServerDefault::RenderingServerDefault  [E:/path/core/os/thread.h @ 90]
    88: _FORCE_INLINE_ ID get_id() const { return id; }
    89: // get the ID of the caller thread
>   90: _FORCE_INLINE_ static ID get_caller_id() { return caller_id; }
    91: // get the ID of the main thread
    92: _FORCE_INLINE_ static ID get_main_id() { return main_thread_id; }
00007FF77A4DE90B 0000021561579CC0 0000021560C1F690 00000215000000FF  godot.windows.opt.tools.64.exe!Main::setup2  [E:/path/main/main.cpp @ 1590]
  1588: /* Initialize Rendering Server */
  1589: 
> 1590: rendering_server = memnew(RenderingServerDefault(OS::get_singleton()->get_render_thread_mode() == OS::RENDER_SEPARATE_THREAD));
  1591: 
  1592: rendering_server->init();
00007FF77A4E479B 0000021560C20C30 0000000000000001 000002155F450860  godot.windows.opt.tools.64.exe!Main::setup  [E:/path/main/main.cpp @ 1388]
  1386: 
  1387: if (p_second_phase) {
> 1388: return setup2();
  1389: }
  1390: 
00007FF77A4C18A7 000002155F153296 0000003CE61FFD4C 0000000000000001  godot.windows.opt.tools.64.exe!widechar_main  [E:/path/platform/windows/godot_windows.cpp @ 151]
   149: TEST_MAIN_PARAM_OVERRIDE(argc, argv_utf8)
   150: 
>  151: Error err = Main::setup(argv_utf8[0], argc - 1, &argv_utf8[1]);
   152: 
   153: if (err != OK) {
00007FF77A4C19A0 0000000000000000 0000000000000047 00007FF7811164D8  godot.windows.opt.tools.64.exe!_main  [E:/path/platform/windows/godot_windows.cpp @ 185]
   183: }
   184: 
>  185: result = widechar_main(argc, wc_argv);
   186: 
   187: LocalFree(wc_argv);
00007FF77A4C13B1 0000000000000000 0000000000000000 0000000000000000  godot.windows.opt.tools.64.exe!__tmainCRTStartup  [C:/_/M/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crtexe.c @ 321]
00007FF77A4C14E6 0000000000000000 0000000000000000 0000000000000000  godot.windows.opt.tools.64.exe!mainCRTStartup  [C:/_/M/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/crt/crtexe.c @ 202]
00007FF8F8997034 0000000000000000 0000000000000000 0000000000000000  KERNEL32.DLL!BaseThreadInitThunk
00007FF8FA1C2651 0000000000000000 0000000000000000 0000000000000000  ntdll.dll!RtlUserThreadStart
Log of the crashing MinGW release_debug build
Godot Engine v4.0.dev.custom_build.bdcc8741e - https://godotengine.org
Using "ICU / HarfBuzz / Graphite" text server...
Vulkan API 1.2.162
ERROR: GENERAL - Message Id Number: 0 | Message Id Name: Loader Message
        loader_get_json: Failed to open JSON file C:\ProgramData\GOG.com\Galaxy\redists\overlay\injected\galaxy_overlay_vklayer_x64.json
        Objects - 1
                Object[0] - VK_OBJECT_TYPE_INSTANCE, Handle 2934791902880
   at: _debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:157)
WARNING: GENERAL - Message Id Number: 0 | Message Id Name: Loader Message
        ReadDataFilesInRegistry: Registry lookup failed to get layer manifest files.
        Objects - 1
                Object[0] - VK_OBJECT_TYPE_INSTANCE, Handle 2934791902880
     at: _debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:154)
Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 2080
- Vulkan multiview supported:
  max view count: 32
  max instances: 134217727
- Vulkan subgroup:
  size: 32
  stages: STAGE_VERTEX, STAGE_TESSELLATION_CONTROL, STAGE_TESSELLATION_EVALUATION, STAGE_GEOMETRY, STAGE_FRAGMENT, STAGE_COMPUTE, STAGE_RAYGEN_KHR, STAGE_ANY_HIT_KHR, STAGE_CLOSEST_HIT_KHR, STAGE_MISS_KHR, STAGE_INTERSECTION_KHR, STAGE_CALLABLE_KHR, STAGE_TASK_NV, STAGE_MESH_NV
  supported ops: FEATURE_BASIC, FEATURE_VOTE, FEATURE_ARITHMETIC, FEATURE_BALLOT, FEATURE_SHUFFLE, FEATURE_SHUFFLE_RELATIVE, FEATURE_CLUSTERED, FEATURE_QUAD, FEATURE_PARTITIONED_NV
  quad operations in all stages
Using present mode: VK_PRESENT_MODE_FIFO_KHR
Using "winink" pen tablet driver...
Log of the working MinGW debug build
Godot Engine v4.0.dev.custom_build.bdcc8741e - https://godotengine.org
Using "ICU / HarfBuzz / Graphite" text server...
Vulkan API 1.2.162
ERROR: GENERAL - Message Id Number: 0 | Message Id Name: Loader Message
        loader_get_json: Failed to open JSON file C:\ProgramData\GOG.com\Galaxy\redists\overlay\injected\galaxy_overlay_vklayer_x64.json
        Objects - 1
                Object[0] - VK_OBJECT_TYPE_INSTANCE, Handle 1835716830080
   at: _debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:157)
WARNING: GENERAL - Message Id Number: 0 | Message Id Name: Loader Message
        ReadDataFilesInRegistry: Registry lookup failed to get layer manifest files.
        Objects - 1
                Object[0] - VK_OBJECT_TYPE_INSTANCE, Handle 1835716830080
     at: _debug_messenger_callback (drivers\vulkan\vulkan_context.cpp:154)
Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 2080
- Vulkan multiview supported:
  max view count: 32
  max instances: 134217727
- Vulkan subgroup:
  size: 32
  stages: STAGE_VERTEX, STAGE_TESSELLATION_CONTROL, STAGE_TESSELLATION_EVALUATION, STAGE_GEOMETRY, STAGE_FRAGMENT, STAGE_COMPUTE, STAGE_RAYGEN_KHR, STAGE_ANY_HIT_KHR, STAGE_CLOSEST_HIT_KHR, STAGE_MISS_KHR, STAGE_INTERSECTION_KHR, STAGE_CALLABLE_KHR, STAGE_TASK_NV, STAGE_MESH_NV
  supported ops: FEATURE_BASIC, FEATURE_VOTE, FEATURE_ARITHMETIC, FEATURE_BALLOT, FEATURE_SHUFFLE, FEATURE_SHUFFLE_RELATIVE, FEATURE_CLUSTERED, FEATURE_QUAD, FEATURE_PARTITIONED_NV
  quad operations in all stages
Using present mode: VK_PRESENT_MODE_FIFO_KHR
Using "winink" pen tablet driver...
Shader 'VoxelGiSdfShaderRD' SHA256: b6972a55781e21cfbac0dfc035381ee1f65e972c1f7e0bcc6004bfb74cc9dc9e
Shader 'ParticlesShaderRD' SHA256: cbb6a5ce1933875a8e2df4f0cde4684286dedb6737cfbcfdeba745c978a2cc45
Shader 'ParticlesCopyShaderRD' SHA256: 6bd2ea6fca4c3a51f1e9bcd5068315e63820ec68fd8596e20480174c943bb9df
Shader 'CanvasSdfShaderRD' SHA256: 692328d90ee7d404503d304e2464af3f3ac466ebd287762fd8883248f3e2a046
Shader 'SkeletonShaderRD' SHA256: 8f03a2ebc895d833120c3b2341c16e971fb1404dd3ae9e8030bc4bbf63e67fd5
Shader 'CanvasShaderRD' SHA256: a30a00a53b17f4e413dfb1fe109868063f2d67900aaeec66b0f50964108e216b
Shader 'CanvasOcclusionShaderRD' SHA256: fd08d5c92b6537852fb4b8e5fba27f28518faad41a5c21ac9bd51bf49807824c
Shader 'ClusterRenderShaderRD' SHA256: a2924afb04064a1dc42ed9d63721e7d5dd0466f245037ff85b5bef8bd251f515
Shader 'ClusterStoreShaderRD' SHA256: 64c437e6bebba9faa4766b524281310d934fd2a28f26e9a8a7834ca07f09d2dc
Shader 'ClusterDebugShaderRD' SHA256: 0f66ac91d81e6f440ffa804d473579cb5d7ac135b375dce6d120186a65b54204
Shader 'SkyShaderRD' SHA256: dd0dbf614e400cc2f58360d292f85f19237fdd9746b9a46c6fd7c50fe8a2ae43
Shader 'VoxelGiShaderRD' SHA256: 51aa13996be8fb6071160d3ea73d752edcd67ef81e1aec1ec3623a13eee94d0a
Shader 'VoxelGiDebugShaderRD' SHA256: 7fa6be49a459f8891a6e756165f2e9ca7f3ba261f53edf36c2041dc2acbf2a96
Shader 'SdfgiPreprocessShaderRD' SHA256: 056a9e6eaae9cc98093ceec6142bcb4582be0f3443e18402c1e25c293899b104
Shader 'SdfgiDirectLightShaderRD' SHA256: 0707e534eb11aeb7566a94913fe168d96b66896b440650f768f92ec6ed5cc842
Shader 'SdfgiIntegrateShaderRD' SHA256: deacba90b0c075f22f4000295f6a0b846957d2602b5d54b631c089792288e7df
Shader 'GiShaderRD' SHA256: c1d65ee198e82b396759752f26b64ae98d7bb8691aed1bc07924617cf25872c5
Shader 'SdfgiDebugShaderRD' SHA256: 5e9f1e4d567be5c04dacec2ddbddb98638086a1fe7dc1d0260f3a58e93856a0e
Shader 'SdfgiDebugProbesShaderRD' SHA256: a6a825191c0869482f513376305eb79fa16189e65068bf76a484a0e1eb602c1b
Shader 'VolumetricFogShaderRD' SHA256: bc8fa35c7d92fd0353d5458af7d19438bf0c46f8c873b1b5e45f328f988f2d34
Shader 'SceneForwardClusteredShaderRD' SHA256: 300732714cbaf090328e341566d4d056b1bf83a2786d31de56778f6c929d1987
Shader 'CopyShaderRD' SHA256: 4ea51535b5cdcba7ece20ebd074657d05cbde47d1d0e87d3c84a2a1c28aa976f
Shader 'CopyToFbShaderRD' SHA256: 6972e6ea6ece204edb2197ac92f699ad4e7c4b22cac23422dd816a2a9fc11ce3
Shader 'CubemapRoughnessShaderRD' SHA256: 083d83f629fde69ed8e4c482638f30df40144b9e05c8d8243317b1866ff97897
Shader 'TonemapShaderRD' SHA256: cd4d7156689aa910864bd876e2c439c8f2b49dd281df707f93145fec99b9bd67
Shader 'LuminanceReduceShaderRD' SHA256: f8db4abaa10d52974aef5a0443e69f8838eb05f4b4eee120c19ca3b0f8ef6353
Shader 'CubeToDpShaderRD' SHA256: 1ac5646aaebfed0f2fa62c6faa2f87981b8a5acf4748191482448d7fe516cc78
Shader 'BokehDofShaderRD' SHA256: 7ae11ecc5f3863fff19a28c9e41a9ba8016c53f54cc182daa15e6a1e080ce3c6
Shader 'SsaoDownsampleShaderRD' SHA256: 4db35346ca9c82436e993cde3466bee499a8f66f6d58e39d9bed78ee7d835e1f
Shader 'SsaoShaderRD' SHA256: db7ede5f05458aaa5e03db5e62804bcd7e3b4b3eeb556e9eb027a958a96b6bd7
Shader 'SsaoImportanceMapShaderRD' SHA256: a2305d0dc9bed6da701e58b6d6a28bb2a664aedb9de4460c390b21a7f989e8eb
Shader 'SsaoBlurShaderRD' SHA256: 9158ebb0e50201b64d1c1009317e7cf5de0792a82a4d06b393e5db5ec351b54f
Shader 'SsaoInterleaveShaderRD' SHA256: e8d5eda46653d75788db067517688755630bcb40ff6da19c5f039db86df234c3
Shader 'RoughnessLimiterShaderRD' SHA256: fe30b9757ac0d3bbb53ab48bc7e0d48cc760f8212fc723c588c30dc9466539d4
Shader 'CubemapDownsamplerShaderRD' SHA256: 8184770cb634e8d5b7f7d4e801b864caa2972a7a98181a62c196ffa7b882edc3
Shader 'CubemapFilterShaderRD' SHA256: 5c27e7fe5c0fce79c147c7a8ae6984a0a7ae2dd90af07e35a4a34934698a91ca
Shader 'SpecularMergeShaderRD' SHA256: f6dc7afac1a2715cc2f134daff1bddfd6b7ca715d655aca8cafa32798395dec2
Shader 'ScreenSpaceReflectionShaderRD' SHA256: d56b77268d5c239c3886ca85201d733c47ea7e6542969c5eb6d5350b483778ef
Shader 'ScreenSpaceReflectionFilterShaderRD' SHA256: 4cd6afe65db3d97e059daa77702faf2a6e6cccb9c552c61af6055cb5be448b5f
Shader 'ScreenSpaceReflectionScaleShaderRD' SHA256: 89c672345ad85d25a8e9cbc814aac4007cb5fcb29cb8262d0185abd23fdccb9f
Shader 'SubsurfaceScatteringShaderRD' SHA256: c633c5beb82a7102c2849bec06732e4c6c98ebfc17b013f0844089d30beaef52
Shader 'ResolveShaderRD' SHA256: d0c9281d4f856dc19c5c6d14e0b256b28ca5455371db93744c010476572dbf25
Shader 'SortShaderRD' SHA256: 2192a5f1b643af9d53a2a609c7d2a715237ea16c083317a1611af42c6845b99c
Shader 'BlitShaderRD' SHA256: 3c3477999355f834dd2a898d94df5b40ae97299bb1f5e9de2dd2b6caf46055f5
WASAPI: wFormatTag = 65534
WASAPI: nChannels = 8
WASAPI: nSamplesPerSec = 48000
WASAPI: nAvgBytesPerSec = 1536000
WASAPI: nBlockAlign = 32
WASAPI: wBitsPerSample = 32
WASAPI: cbSize = 22
WASAPI: detected 8 channels
WASAPI: audio buffer frames: 1962 calculated latency: 44ms

ERROR: Attempted to free invalid ID: 2100239007746
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
CORE API HASH: 12701809910231588744
EDITOR API HASH: 7657489752925521230
Using present mode: VK_PRESENT_MODE_FIFO_KHR
Loaded builtin certs
EditorSettings: Save OK!
Using present mode: VK_PRESENT_MODE_FIFO_KHR
Using present mode: VK_PRESENT_MODE_FIFO_KHR
Using present mode: VK_PRESENT_MODE_FIFO_KHR
EditorSettings: Save OK!
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: Attempted to free invalid ID: 0
   at: _free_internal (drivers\vulkan\rendering_device_vulkan.cpp:8313)
ERROR: 1 RID allocations of type 'N17RendererStorageRD7TextureE' were leaked at exit.
WARNING: 2 RIDs of type "Texture" were leaked.
     at: finalize (drivers\vulkan\rendering_device_vulkan.cpp:9055)
StringName: 173 unclaimed string names at exit.

Summary of tried combinations and result of it:

Distribution MinGW version GCC version binutils version Works
Fedora 34 8.0.0 10.3.1 2.34 yes
Fedora 35 9.0.0 11.2.1 2.37 no
Debian Sid 8.0.0 10.2.1 2.37 no
OpenSuse TW 9.0.0 9.2.0 2.33 yes
MSYS2 9.0.0.6346.6cc97775a-1 11.2.0 2.37 no
Mageia 8 8.0.0 10.2.1 2.34 yes[1]
MSYS2 9.0.0.6454.b4445ee52 11.2.0 2.38 no
MSYS2 10.0.0.r0.gaa08f56da-1 11.2.0 2.38 no

[1] The icon from the executable was missing

Steps to reproduce

For a not working build:
scons platform=windows target=release_debug use_mingw=yes -j12

For a working build:
scons platform=windows target=debug use_mingw=yes -j12

Minimal reproduction project

No response

@akien-mga
Copy link
Member

There's likely the same issue in the 3.x branch, which has been exposed by the first 3.4 beta builds: #50970 (comment)

3.3.x builds already include the Thread modernization work, but they're built on Fedora 33 with MinGW 7.0.0 and GCC 10.2.1.

The new builds (3.4 beta 1 and beta 2, and https://downloads.tuxfamily.org/godotengine/testing/4.0/4.0-dev.20210727/) are built on Fedora 34 with MinGW 8.0.0 and GCC 10.3.1.

@MmAaXx500
Copy link
Contributor Author

@akien-mga I have just built the 3.x and it's working, it's not crashing when opening the editor. (v3.4.beta.custom_build.7075bb612)
Build from https://downloads.tuxfamily.org/godotengine/testing/4.0/4.0-dev.20210727/ is working too.

@akien-mga
Copy link
Member

Thanks, these might be separate issues in the end then.

Which MinGW versions do you have (both the version of MinGW itself, which should typically be 7.0.0, 8.0.0 or 9.0.0, and the GCC or LLVM version compiled against the MinGW headers)?

@MmAaXx500
Copy link
Contributor Author

I'm using MSYS with MinGW 9.0.0.6246.ae63cde27-1 and GCC 10.3.0.

gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-10.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --enable-checking=release --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++,jit --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --enable-lto --enable-libgomp --disable-multilib --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev5, Built by MSYS2 project' --with-bugurl=https://github.com/msys2/MINGW-packages/issues --with-gnu-as --with-gnu-ld --with-boot-ldflags='-pipe -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high -Wl,--disable-dynamicbase -static-libstdc++ -static-libgcc' 'LDFLAGS_FOR_TARGET=-pipe -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high' --enable-linker-plugin-flags='LDFLAGS=-static-libstdc++\ -static-libgcc\ -pipe\ -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high\ -Wl,--stack,12582912'
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.3.0 (Rev5, Built by MSYS2 project)

@MmAaXx500
Copy link
Contributor Author

I tested it again to see if it was still relevant. I tried with 0121ce9 and with updated MSYS and MinGW, but the result is exactly same.

MinGW: 9.0.0.6346.6cc97775a-1
GCC: 11.2.0

gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-11.2.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --enable-checking=release --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++,jit --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts --enable-libstdcxx-time --disable-libstdcxx-pch --disable-libstdcxx-debug --enable-lto --enable-libgomp --disable-multilib --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://github.com/msys2/MINGW-packages/issues --with-gnu-as --with-gnu-ld --with-boot-ldflags='-pipe -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high -Wl,--disable-dynamicbase -static-libstdc++ -static-libgcc' 'LDFLAGS_FOR_TARGET=-pipe -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high' --enable-linker-plugin-flags='LDFLAGS=-static-libstdc++\ -static-libgcc\ -pipe\ -Wl,--dynamicbase,--high-entropy-va,--nxcompat,--default-image-base-high\ -Wl,--stack,12582912'
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.2.0 (Rev2, Built by MSYS2 project)

@MmAaXx500
Copy link
Contributor Author

A few more things I found out:
If I compile with mingw from OpenSuse it's works. It has gcc 9.2.0.
I tried with mingw from Debian testing, but I have no success with it. It has gcc 10.

I believe (but cannot prove) that there are compatibility problems with the newer versions of the compiler.

OpenSuse gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-w64-mingw32/9.2.0/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../configure --prefix=/usr --bindir=/usr/bin --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/lib64 --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --build=x86_64-suse-linux-gnu --host=x86_64-suse-linux-gnu --target=x86_64-w64-mingw32 --with-gnu-as --with-gnu-ld --verbose --without-newlib --disable-multilib --enable-shared --disable-plugin --with-system-zlib --disable-nls --without-included-gettext --disable-win32-registry --enable-threads=posix --enable-version-specific-runtime-libs --with-sysroot=/usr/x86_64-w64-mingw32/sys-root --enable-languages=c,c++,fortran,objc,obj-c++ --without-x --enable-hash-synchronization --enable-fully-dynamic-string --enable-libgomp --enable-linker-build-id --disable-vtable-verify --with-pkgversion='SUSE Linux'
Thread model: posix
gcc version 9.2.0 (SUSE Linux)
Debian gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-w64-mingw32/10-posix/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../../src/configure --build=x86_64-linux-gnu --prefix=/usr --includedir='/usr/include' --mandir='/usr/share/man' --infodir='/usr/share/info' --sysconfdir=/etc --localstatedir=/var --disable-option-checking --disable-silent-rules --libdir='/usr/lib/x86_64-linux-gnu' --libexecdir='/usr/lib/x86_64-linux-gnu' --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --enable-shared --enable-static --disable-multilib --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --libdir=/usr/lib --enable-libstdcxx-time=yes --with-tune=generic --with-headers --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libgomp --enable-languages=c,c++,fortran,objc,obj-c++,ada --enable-lto --enable-threads=posix --program-suffix=-posix --program-prefix=x86_64-w64-mingw32- --target=x86_64-w64-mingw32 --with-as=/usr/bin/x86_64-w64-mingw32-as --with-ld=/usr/bin/x86_64-w64-mingw32-ld --enable-libatomic --enable-libstdcxx-filesystem-ts=yes --enable-dependency-tracking
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10-posix 20210110 (GCC)

@Calinou
Copy link
Member

Calinou commented Nov 29, 2021

@MmAaXx500 I can compile Godot's master branch just fine with GCC 11.2 provided in the Fedora repositories. This is likely specific to MinGW.

Maybe give llvm-mingw a try? It's still MinGW, but it uses Clang instead of GCC and should provide faster link times. Make sure to add two the installation's bin/ folders to your user's PATH environment variable, then open a new command prompt and use scons use_mingw=yes use_llvm=yes use_lld=yes to compile Godot.

@MmAaXx500
Copy link
Contributor Author

@Calinou Thanks for the suggestion about Clang. I will use it while this gets sorted out.

I can compile Godot's master branch just fine with GCC 11.2 provided in the Fedora repositories. This is likely specific to MinGW.

I can compile it too, but cannot launch. Here is a gif about the issue.

I read in the docs that gcc is recommended for release build, that's why I'm trying to get it working.

What is your opinion, is it a viable option to use Clang for release builds?

@akien-mga
Copy link
Member

Clang might be a decent option for release builds on Windows too, though I don't have much experience with it (official builds use GCC as provided in Fedora 34 - at least an image from 5 months ago).

I'd still be interested to figure out what's the exact component that causes the issue you're experiencing, so we can make sure it won't affect official builds too.

@MmAaXx500
Copy link
Contributor Author

MmAaXx500 commented Nov 30, 2021

Tried with Fedora 34, it's works as expected, but with Fedora 35 I can reproduce this issue. (so do not update the official build server 😄)
Fedora 34 has gcc 10.3.1
Fedora 35 has gcc 11.2.1

Fedora 34 gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-w64-mingw32/10.3.1/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../configure --prefix=/usr --bindir=/usr/bin --includedir=/usr/include --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --with-gnu-as --with-gnu-ld --verbose --without-newlib --disable-multilib --disable-plugin --with-system-zlib --disable-nls --without-included-gettext --disable-win32-registry --enable-languages=c,c++,objc,obj-c++,fortran --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-threads=posix --with-isl --enable-libgomp --target=x86_64-w64-mingw32 --with-sysroot=/usr/x86_64-w64-mingw32/sys-root --with-gxx-include-dir=/usr/x86_64-w64-mingw32/sys-root/mingw/include/c++
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.3.1 20210422 (Fedora MinGW 10.3.1-2.fc34) (GCC)
Fedora 35 gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-w64-mingw32/11.2.1/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../configure --prefix=/usr --bindir=/usr/bin --includedir=/usr/include --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --with-gnu-as --with-gnu-ld --verbose --without-newlib --disable-multilib --disable-plugin --with-system-zlib --disable-nls --without-included-gettext --disable-win32-registry --enable-languages=c,c++,objc,obj-c++,fortran --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-threads=posix --with-isl --enable-libgomp --target=x86_64-w64-mingw32 --with-sysroot=/usr/x86_64-w64-mingw32/sys-root --with-gxx-include-dir=/usr/x86_64-w64-mingw32/sys-root/mingw/include/c++
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 11.2.1 20210728 (Fedora MinGW 11.2.1-3.fc35) (GCC)

@MmAaXx500
Copy link
Contributor Author

MmAaXx500 commented Nov 30, 2021

I think I found something. F34 uses 8.0.0 and F35 uses 9.0.0 mingw (assuming mingw64-headers package version reflects mingw version).
But, in order to make the situation less clear, OpenSuse TW which uses 9.0.0 mingw (mingw64-headers) it runs fine, but Debian uses 8.0.0 mingw (mingw-w64-common) and not works.

Just to summarize, here is a table:
(at mingw version, I'm assuming package version reflects this)

Distro MinGW version GCC version Works
Fedora 34 8.0.0 10.3.1 yes
Fedora 35 9.0.0 11.2.1 no
Debian Sid 8.0.0 10.2.1 no
OpenSuse TW 9.0.0 9.2.0 yes
MSYS2 9.0.0 11.2.0 no

(after I wrote the table, I'm not sure about I really found something)

@akien-mga
Copy link
Member

To clarify, Debian Sid's GCC should be 10.2.1 as of the current package https://packages.debian.org/sid/gcc-mingw-w64

(Though GCC versioning in distros is tricky as .1 patch releases are not releases, it just means "any Git snapshot of the next minor branch", i.e. 10.2.1 can be any commit between 10.2 and 10.3.)

@MmAaXx500
Copy link
Contributor Author

Thanks! Corrected it. I wrote "10" because x86_64-w64-mingw32-gcc -v outputs this: gcc version 10-posix 20210110 (GCC)

@akien-mga
Copy link
Member

I tried a local build on Mageia 9 with MinGW 9.0.0 and GCC 11.2.1 (20211019) which should be pretty much the same as Fedora 35's config as Mageia's mingw-gcc packaging is based on Fedora's.

It works fine for me when running through Wine on Linux, but I have yet to actually test it on Windows (will do asap).

Here's the build if you want to test (note: built with scons p=windows target=release_debug, then stripped with mingw-strip): https://downloads.tuxfamily.org/godotengine/testing/4.0-dev-2021130-mga9-mingw9-gcc11.2.1.zip

@MmAaXx500
Copy link
Contributor Author

Your build works for me.

I built one on Mageia 8 because they don't have a docker image for 9, and it worked too (except it did not have an icon).
Mageia 8 has MinGW 8.0.0 and GCC 10.2.1. (added to the table and moved it to the first post)

gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-w64-mingw32/10.2.1/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../configure --prefix=/usr --bindir=/usr/bin --includedir=/usr/include --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --build=x86_64-mageia-linux-gnu --host=x86_64-mageia-linux-gnu --with-gnu-as --with-gnu-ld --verbose --without-newlib --disable-multilib --disable-plugin --with-system-zlib --disable-nls --without-included-gettext --disable-win32-registry --enable-languages=c,c++,objc,obj-c++,fortran --with-bugurl=https://bugs.mageia.org/ --enable-threads=posix --enable-libgomp --target=x86_64-w64-mingw32 --with-sysroot=/usr/x86_64-w64-mingw32/sys-root --with-gxx-include-dir=/usr/x86_64-w64-mingw32/sys-root/mingw/include/c++
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.2.1 20200723 (Fedora MinGW 10.2.1-2.mga8) (GCC)

@akien-mga
Copy link
Member

That's interesting, then the culprit is likely neither MinGW nor GCC but another component. This could be the (mingw-)binutils version.

Fedora 34, Mageia 8 and Mageia 9 have mingw-binutils 2.34, while Fedora 35 and Debian Sid have mingw-binutils 2.37.
What version does MSYS2 provide?

@MmAaXx500
Copy link
Contributor Author

MSYS2 have 2.37

@akien-mga
Copy link
Member

So yeah binutils sounds like a likely culprit. I'll see if I can use Fedora 35's mingw-binutils on Fedora 34 to see if that produces a crashing binary. And then we'd have to try to bisect that with intermediate releases and report the issue upstream, though we'd need to identify a bit more precisely how binutils relates to this (thread-related?) crash.

@akien-mga
Copy link
Member

Can you still reproduce this with binutils 2.38, if MSYS2 provides it?

@MmAaXx500
Copy link
Contributor Author

Yes I can.
Current versions are:

  • GCC 11.2.0
  • MinGW 9.0.0.6454.b4445ee52
  • binutils 2.38

I wonder why no one has reported this in the last half year. Am I alone with this issue?
Anyway, here is a build for anyone who wants to test/examine it.
https://drive.google.com/file/d/1iLaTmZeH57baIhL3O2FXFs-_nbQl2Trp/view?usp=sharing

@MmAaXx500
Copy link
Contributor Author

Another test because MinGW 10 was released, but the result has not changed.

  • GCC 11.2.0
  • MinGW 10.0.0.r0.gaa08f56da-1
  • binutils 2.38

I have also updated the table in the first post with the new tests and added the binutils version column.

@MmAaXx500
Copy link
Contributor Author

MmAaXx500 commented May 8, 2022

@akien-mga I have some good news!
I have bisected the binutils and found out this is the first bad commit: https://sourceware.org/git/?p=binutils-gdb.git;h=514b4e191d5f46de8e142fe216e677a35fa9c4bb
It was first included in 2.36.

I found a way* to make it work. (* two ways. See follow-up comment) Compile binutils with reverted NT_EXE_IMAGE_BASE and use link flags to get correct result.
Scons command:

scons -j12 platform=windows bits=64 target=release_debug LINKFLAGS="-Wl,--disable-dynamicbase -Wl,--disable-high-entropy-va"

Patch here: MmAaXx500/binutils-gdb-godot@b02b583

Below, some of my inexperienced observations:

  • The --disable-high-entropy-va is redundant because the --disable-dynamicbase disables this as well. (this is a fact, not an observation: source)

  • The flags are required because the above-mentioned commit changes the default dll characteristics from 0 to enable dynamic base, high entropy va and nx compat. The nx compat flag left out from the command on purpose because it does not affect the binary in a way related to this issue.

  • I think this issue is related somehow to the (dynamic) base address, because if I do not revert the NT_EXE_IMAGE_BASE and disable the dynamic base or do the revert but do not disable the dynamic base the result is the same: this issue

I'd like to note that I don't have much knowledge in this area, so I cannot determine who is "at fault" or what exactly causing this issue.
So, I highly appreciate if someone can dig deeper in that and found out the root cause of the issue.

@MmAaXx500
Copy link
Contributor Author

Another workaround without modifying binutils. Just pass the base address as a parameter.

scons -j12 platform=windows bits=64 target=release_debug LINKFLAGS="-Wl,--disable-dynamicbase -Wl,--disable-high-entropy-va -Wl,--image-base=0x400000"

@akien-mga
Copy link
Member

Thanks for documenting all this! That's an area with @hpvb is probably the most competent to figure out what we need to do exactly / if we have to report a regression upstream.

@MmAaXx500
Copy link
Contributor Author

With the release of the alpha 8 I noticed that the Windows binaries works, although it was compiled on Fedora 36 which has binutils 2.37. This is not in line with the previous assumptions/findings, so I tested again on various distributions.

  • Fedora 36 (2.37): Works
  • Debian Sid (2.37): Works
  • MSYS2 (2.38): Works
  • Arch (2.38) with gcc 11.2.0-1: Not works
  • Arch (2.38) with gcc 12.1.0-1: Works

I have looked through the patches of MSYS and the only patch could be related is this, but it was released on 2021 Jul 28, so it is included in all of my test with MSYS (therefore not related). Another thing is the whole binutils package haven't been touched since 2022 Mar 1, the build didn't work on 2022 Apr 20, but it magically started working today (I tested it today, so I do not know when precisely).

Arch currently have mingw-w64-gcc with version 11.2.0-1 in Community and have 12.1.0-1 in Community-Testing. If I compiled and installed the newer gcc from Community-Testing, the issue went away.

I think this issue is neither binutils' nor gcc's solely fault, rather something of a compatibility issue between them, which seems to be resolving itself as the distributions releasing the newer packages.

@akien-mga
Copy link
Member

akien-mga commented May 20, 2022

So after spending some time chasing the wrong issue in #61176, I finally realized that Mono builds for Godot 3.x seem to be affected by the same issue you describe. Windows binaries built on Fedora 35 are crashing, while Fedora 36 is fine.

Thanks a lot for all the debugging you've done so far and documenting your findings. I'm still not sure what we can do to get this actually fixed in the upstream MinGW distributions, but at least we can make sure to use Fedora 36+ / skip Fedora 35 for official builds. Since the bug is fixed in recent toolchains, we can assume that upstream projects did find the regression and got it fixed in the relevant affected projects (but that didn't get backported all the way to past releases).

For the record, I did a couple test builds to confirm that I get a crash with Fedora 35 and a working binary with Fedora 36 in the current master branch:

@akien-mga
Copy link
Member

For the record, I now think that this issue might have been a GCC regression in 11.2 (maybe 11.1 too), which got fixed in later versions. Because that seems to be the only significant between the mingw-headers, mingw-gcc and mingw-binutils package of Fedora 35 and Fedora 36.

So I suspect this was found and fixed in the 11.x stable branch after 11.2 was released. There was a 11.3 release earlier this year which should also have the fix I suppose.

TL;DR: Avoid GCC 11.1 or 11.2, unless it's a snapshot more recent than the July 2021 11.2 release. GCC 11.3 or 12.1+ should be fine.

@akien-mga akien-mga changed the title Godot 4 compiled with MinGW is crashing with a white screen Godot 4 compiled with MinGW is crashing with a white screen [GCC 11.x bug fixed in 11.3] Jun 10, 2022
@akien-mga
Copy link
Member

Closing as there isn't much more actionable from our side, aside from having this documented here. We could put a note in the compilation docs or add some checks to prevent building with GCC 11.1 or 11.2, but most distros by now should provide either 11.3, a newer GCC, or still be on an older version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants