Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MingW build broke with latest MingW #641

Closed
MarkCallow opened this issue Oct 20, 2022 · 14 comments · Fixed by #642
Closed

MingW build broke with latest MingW #641

MarkCallow opened this issue Oct 20, 2022 · 14 comments · Fixed by #642

Comments

@MarkCallow
Copy link
Collaborator

MarkCallow commented Oct 20, 2022 •
edited

The MingW build has been broken by an update to the MingW installation on GitHub Actions.

The issue is at

KTX-Software/lib/filestream.c

Lines 310 to 317 in f4feff2

// Need to flush so that fstat will return the current size.
// Can ignore return value. The only error that can happen is to tell you
// it was a NOP because the file is read only.
#if (defined(_MSC_VER) && _MSC_VER < 1900) || defined(__MINGW32__)
// Bug in VS2013 msvcrt. fflush on FILE open for READ changes file offset
// to 4096.
if (str->data.file->_flag & _IOWRT)
#endif

The GCC version in the MingW where this works is 11.2. That in the version where it is broken is 12.2. I do not know the MingW version as cmake config does not report it. Presumably the later version is no longer using the msvcrt that had the problem. The obvious fix is to remove the || defined(__MINGW32__) but that will break running, but not building, in earlier versions of MingW.

@Honeybunch please provide a PR to fix this. I do not have a MingW installation so can't test a fix especially not on multiple versions. Presumably a fix that will not break earlier versions will involve using the __MINGWXX_MAJOR_VERSION__ and ``MINGWXX_MAJOR_VERSION` defines.

Why are we using __MINGW32__ in the above and not __MINGW64__ We should be building 64-bit libraries and applications so the latter seems more appropriate.
@Honeybunch
Copy link
Contributor

Oh no! I'll take a look soon

@MarkCallow
Copy link
Collaborator Author

Great. I am ready to make Release 4.1.0 and this cropped up as I was updating the code-signing part of the build.

I'm also getting warnings about node.js 12 actions being deprecated. This affects the seanmiddleditch ninja action. Probably the reference needs updating to a newer version. This build shows the issues.

@Honeybunch
Copy link
Contributor

I have a repro and a local build fix but it seems like there are some serious test failures. I'll need to attach a debugger and hunt those down. This seems to happen regardless of whether or not I try to specify linking the ucrt or msvcrt. Though I may be messing that bit up.

[ctest] 69% tests passed, 101 tests failed out of 331
[ctest] 
[ctest] Total Test time (real) =   4.40 sec
[ctest] 
[ctest] The following tests FAILED:
[ctest] 	 59 - texturetestktxTexture2_IterateLevelFacesTest.InvalidValueOnNullCallback (Exit code 0xc0000409
[ctest] )
[ctest] 	 60 - texturetestktxTexture2_IterateLevelFacesTest.IterateImages (Exit code 0xc0000409
[ctest] )
[ctest] 	 61 - texturetestktxTexture2_IterateLevelsTest.InvalidValueOnNullCallback (Exit code 0xc0000409
[ctest] )
[ctest] 	 62 - texturetestktxTexture2_IterateLevelsTest.IterateLevels (Exit code 0xc0000409
[ctest] )
[ctest] 	 67 - texturetestktxTexture2_LoadImageDataTest.InvalidOpWhenDataAlreadyLoaded (Exit code 0xc0000409
[ctest] )
[ctest] 	 68 - texturetestktxTexture2_LoadImageDataTest.InvalidOpWhenDataAlreadyLoadedToExternal (Exit code 0xc0000409
[ctest] )
[ctest] 	 69 - texturetestktxTexture2_LoadImageDataTest.LoadImageDataInternal (Exit code 0xc0000409
[ctest] )
[ctest] 	 70 - texturetestktxTexture2_LoadImageDataTest.LoadImageDataExternal (Exit code 0xc0000409
[ctest] )
[ctest] 	 71 - texturetestktxTexture2_CreateCopyTest.CreateCopy (Exit code 0xc0000409
[ctest] )
[ctest] 	127 - texturetestktxTexture2_BasisCompressTest.Compress (Exit code 0xc0000409
[ctest] )
[ctest] 	128 - texturetestktxTexture2_GetNumComponentsTestR8.Uncompressed (Exit code 0xc0000409
[ctest] )
[ctest] 	129 - texturetestktxTexture2_GetNumComponentsTestR8.BasisLZ (Exit code 0xc0000409
[ctest] )
[ctest] 	130 - texturetestktxTexture2_GetNumComponentsTestR8.UASTC (Exit code 0xc0000409
[ctest] )
[ctest] 	131 - texturetestktxTexture2_GetNumComponentsTestRG8.Uncompressed (Exit code 0xc0000409
[ctest] )
[ctest] 	132 - texturetestktxTexture2_GetNumComponentsTestRG8.BasisLZ (Exit code 0xc0000409
[ctest] )
[ctest] 	133 - texturetestktxTexture2_GetNumComponentsTestRG8.UASTC (Exit code 0xc0000409
[ctest] )
[ctest] 	134 - texturetestktxTexture2_GetNumComponentsTestRGB8.Uncompressed (Exit code 0xc0000409
[ctest] )
[ctest] 	135 - texturetestktxTexture2_GetNumComponentsTestRGB8.BasisLZ (Exit code 0xc0000409
[ctest] )
[ctest] 	136 - texturetestktxTexture2_GetNumComponentsTestRGB8.UASTC (Exit code 0xc0000409
[ctest] )
[ctest] 	137 - texturetestktxTexture2_GetNumComponentsTestRGBA8.Uncompressed (Exit code 0xc0000409
[ctest] )
[ctest] 	138 - texturetestktxTexture2_GetNumComponentsTestRGBA8.BasisLZ (Exit code 0xc0000409
[ctest] )
[ctest] 	139 - texturetestktxTexture2_GetNumComponentsTestRGBA8.UASTC (Exit code 0xc0000409
[ctest] )
[ctest] 	140 - texturetestktxTexture2_MetadataTest.EmptyValue (Exit code 0xc0000409
[ctest] )
[ctest] 	141 - texturetestktxTexture2_MetadataTest.NoLibVersionDupOnMultipleWrites (Exit code 0xc0000409
[ctest] )
[ctest] 	142 - texturetestktxTexture2_MetadataTest.LibVersionUpdatedCorrectly (Exit code 0xc0000409
[ctest] )
[ctest] 	148 - ktx2check-test-all (Failed)
[ctest] 	150 - ktx2check-test-stdin-read (Failed)
[ctest] 	151 - ktx2check-test-pipe-read (Failed)
[ctest] 	171 - ktx2ktx2-cnvrt-2d-uncompressed (Failed)
[ctest] 	172 - ktx2ktx2-cnvrt-2d-bc2 (Failed)
[ctest] 	173 - ktx2ktx2-cnvrt-2d-array-astc (Failed)
[ctest] 	174 - ktx2ktx2-cnvrt-implied-out-2d-bc2 (Failed)
[ctest] 	183 - ktxsc-zcmp-cubemap (Failed)
[ctest] 	184 - ktxsc-inplace-curdir-zcmp-cubemap (Failed)
[ctest] 	185 - ktxsc-inplace-diffdir-zcmp_cubemap (Failed)
[ctest] 	210 - toktx-cmp-rgb-reference (Failed)
[ctest] 	211 - toktx-cmp-rgb-amg-reference (Failed)
[ctest] 	212 - toktx-cmp-orient-up (Failed)
[ctest] 	213 - toktx-cmp-orient-up-metadata (Failed)
[ctest] 	214 - toktx-cmp-orient-down-metadata (Failed)
[ctest] 	215 - toktx-cmp-rgba-reference (Failed)
[ctest] 	216 - toktx-cmp-rgb-mipmap-reference (Failed)
[ctest] 	217 - toktx-cmp-rgb-mipmap-reference-u (Failed)
[ctest] 	218 - toktx-cmp-cimg5293_uastc (Failed)
[ctest] 	219 - toktx-cmp-cimg5293_uastc_zstd (Failed)
[ctest] 	220 - toktx-cmp-16bit_png_uastc (Failed)
[ctest] 	221 - toktx-cmp-luminance_reference_u (Failed)
[ctest] 	222 - toktx-cmp-luminance_reference_uastc (Failed)
[ctest] 	223 - toktx-cmp-luminance_reference_basis (Failed)
[ctest] 	224 - toktx-cmp-luminance_alpha_reference_u (Failed)
[ctest] 	225 - toktx-cmp-luminance_alpha_reference_uastc (Failed)
[ctest] 	226 - toktx-cmp-r_reference_u (Failed)
[ctest] 	227 - toktx-cmp-r_reference_uastc (Failed)
[ctest] 	228 - toktx-cmp-r_reference_basis (Failed)
[ctest] 	229 - toktx-cmp-rg_reference_u (Failed)
[ctest] 	230 - toktx-cmp-rg_reference_uastc (Failed)
[ctest] 	231 - toktx-cmp-swizzle_r_to_g_u (Failed)
[ctest] 	232 - toktx-cmp-swizzle_r_to_gb_convert_to_rgba_u (Failed)
[ctest] 	233 - toktx-cmp-swizzle_r_to_gb_convert_to_rgba_basis (Failed)
[ctest] 	234 - toktx-cmp-swizzle_r_to_gb_convert_to_rgba_uastc (Failed)
[ctest] 	235 - toktx-cmp-uastc_Iron_Bars_001_normal (Failed)
[ctest] 	236 - toktx-cmp-gAMA_chunk_png (Failed)
[ctest] 	237 - toktx-cmp-cHRM_chunk_png (Failed)
[ctest] 	238 - toktx-cmp-tRNS_chunk_rgb_png (Failed)
[ctest] 	239 - toktx-cmp-tRNS_chunk_palette_8-bit_png (Failed)
[ctest] 	240 - toktx-cmp-tRNS_chunk_palette_2-bit_png (Failed)
[ctest] 	243 - toktx-cmp-astc_mipmap_ldr_cubemap_6x6 (Failed)
[ctest] 	244 - toktx-cmp-astc_ldr_cubemap_6x6 (Failed)
[ctest] 	245 - toktx-cmp-astc_mipmap_ldr_6x6_posx (Failed)
[ctest] 	246 - toktx-cmp-astc_ldr_6x6_posx (Failed)
[ctest] 	247 - toktx-cmp-astc_mipmap_ldr_6x6_posz (Failed)
[ctest] 	248 - toktx-cmp-astc_mipmap_ldr_6x6_posy (Failed)
[ctest] 	249 - toktx-cmp-astc_mipmap_ldr_6x6_kodim17_fastest (Failed)
[ctest] 	250 - toktx-cmp-astc_mipmap_ldr_6x6_kodim17_fast (Failed)
[ctest] 	251 - toktx-cmp-astc_mipmap_ldr_6x6_kodim17_medium (Failed)
[ctest] 	252 - toktx-cmp-astc_mipmap_ldr_4x4_posx (Failed)
[ctest] 	253 - toktx-cmp-astc_mipmap_ldr_6x5_posx (Failed)
[ctest] 	254 - toktx-cmp-astc_mipmap_ldr_8x6_posx (Failed)
[ctest] 	255 - toktx-cmp-astc_mipmap_ldr_10x5_posx (Failed)
[ctest] 	256 - toktx-cmp-astc_mipmap_ldr_8x8_posx (Failed)
[ctest] 	257 - toktx-cmp-astc_mipmap_ldr_12x10_posx (Failed)
[ctest] 	258 - toktx-cmp-astc_mipmap_ldr_12x12_posx (Failed)
[ctest] 	259 - toktx-cmp-astc_ldr_4x4_FlightHelmet_baseColor (Failed)
[ctest] 	260 - toktx-cmp-astc_ldr_6x5_FlightHelmet_baseColor (Failed)
[ctest] 	261 - toktx-cmp-astc_ldr_8x6_FlightHelmet_baseColor (Failed)
[ctest] 	262 - toktx-cmp-astc_ldr_10x5_FlightHelmet_baseColor (Failed)
[ctest] 	263 - toktx-cmp-astc_ldr_8x8_FlightHelmet_baseColor (Failed)
[ctest] 	264 - toktx-cmp-astc_ldr_12x10_FlightHelmet_baseColor (Failed)
[ctest] 	265 - toktx-cmp-astc_ldr_12x12_FlightHelmet_baseColor (Failed)
[ctest] 	266 - toktx-cmp-astc_ldr_6x6_Iron_Bars_001_normal (Failed)
[ctest] 	267 - toktx-cmp-astc_ldr_5x4_Iron_Bars_001_normal (Failed)
[ctest] 	268 - toktx-cmp-astc_ldr_6x6_arraytex_7 (Failed)
[ctest] 	269 - toktx-cmp-astc_ldr_6x6_arraytex_7_mipmap (Failed)
[ctest] 	270 - toktx-cmp-astc_ldr_6x6_3dtex_7 (Failed)
[ctest] 	271 - toktx-cmp-3dtex_1_reference_u (Failed)
[ctest] 	272 - toktx-cmp-3dtex_7_reference_u (Failed)
[ctest] 	273 - toktx-cmp-arraytex_1_reference_u (Failed)
[ctest] 	274 - toktx-cmp-arraytex_7_reference_u (Failed)
[ctest] 	275 - toktx-cmp-arraytex_7_mipmap_reference_u (Failed)
[ctest] 	330 - streamtestktxStreamTest.CanWriteKtx1AsKtx2ToCppStream (Exit code 0xc0000409
[ctest] )
[ctest] 	331 - streamtestktxStreamTest.CanWriteKtx2ToCppStream (Exit code 0xc0000409
[ctest] )

@Honeybunch
Copy link
Contributor

This all seemed strange so on a different machine I tried building with a different mingw-w64 distribution. This time from https://github.com/skeeto/w64devkit instead of chocolatey. It doesn't have any problems building but does seem to exhibit similar test failures

@Honeybunch
Copy link
Contributor

Honeybunch commented Oct 20, 2022 •
edited

It's a different set of failing tests actually

79% tests passed, 70 tests failed out of 331

Total Test time (real) = 122.61 sec

The following tests FAILED:
         59 - texturetestktxTexture2_IterateLevelFacesTest.InvalidValueOnNullCallback (Failed)
         60 - texturetestktxTexture2_IterateLevelFacesTest.IterateImages (Failed)
         61 - texturetestktxTexture2_IterateLevelsTest.InvalidValueOnNullCallback (Failed)
         62 - texturetestktxTexture2_IterateLevelsTest.IterateLevels (Failed)
         67 - texturetestktxTexture2_LoadImageDataTest.InvalidOpWhenDataAlreadyLoaded (Failed)
         68 - texturetestktxTexture2_LoadImageDataTest.InvalidOpWhenDataAlreadyLoadedToExternal (Failed)
         69 - texturetestktxTexture2_LoadImageDataTest.LoadImageDataInternal (Failed)
         70 - texturetestktxTexture2_LoadImageDataTest.LoadImageDataExternal (Failed)
         71 - texturetestktxTexture2_CreateCopyTest.CreateCopy (Failed)
        127 - texturetestktxTexture2_BasisCompressTest.Compress (Failed)
        128 - texturetestktxTexture2_GetNumComponentsTestR8.Uncompressed (Failed)
        129 - texturetestktxTexture2_GetNumComponentsTestR8.BasisLZ (Failed)
        130 - texturetestktxTexture2_GetNumComponentsTestR8.UASTC (Failed)
        131 - texturetestktxTexture2_GetNumComponentsTestRG8.Uncompressed (Failed)
        132 - texturetestktxTexture2_GetNumComponentsTestRG8.BasisLZ (Failed)
        133 - texturetestktxTexture2_GetNumComponentsTestRG8.UASTC (Failed)
        134 - texturetestktxTexture2_GetNumComponentsTestRGB8.Uncompressed (Failed)
        135 - texturetestktxTexture2_GetNumComponentsTestRGB8.BasisLZ (Failed)
        136 - texturetestktxTexture2_GetNumComponentsTestRGB8.UASTC (Failed)
        137 - texturetestktxTexture2_GetNumComponentsTestRGBA8.Uncompressed (Failed)
        138 - texturetestktxTexture2_GetNumComponentsTestRGBA8.BasisLZ (Failed)
        139 - texturetestktxTexture2_GetNumComponentsTestRGBA8.UASTC (Failed)
        140 - texturetestktxTexture2_MetadataTest.EmptyValue (Failed)
        141 - texturetestktxTexture2_MetadataTest.NoLibVersionDupOnMultipleWrites (Failed)
        142 - texturetestktxTexture2_MetadataTest.LibVersionUpdatedCorrectly (Failed)
        148 - ktx2check-test-all (Failed)
        183 - ktxsc-zcmp-cubemap (Failed)
        184 - ktxsc-inplace-curdir-zcmp-cubemap (Failed)
        185 - ktxsc-inplace-diffdir-zcmp_cubemap (Failed)
        218 - toktx-cmp-cimg5293_uastc (Failed)
        219 - toktx-cmp-cimg5293_uastc_zstd (Failed)
        220 - toktx-cmp-16bit_png_uastc (Failed)
        222 - toktx-cmp-luminance_reference_uastc (Failed)
        223 - toktx-cmp-luminance_reference_basis (Failed)
        225 - toktx-cmp-luminance_alpha_reference_uastc (Failed)
        227 - toktx-cmp-r_reference_uastc (Failed)
        228 - toktx-cmp-r_reference_basis (Failed)
        230 - toktx-cmp-rg_reference_uastc (Failed)
        233 - toktx-cmp-swizzle_r_to_gb_convert_to_rgba_basis (Failed)
        234 - toktx-cmp-swizzle_r_to_gb_convert_to_rgba_uastc (Failed)
        235 - toktx-cmp-uastc_Iron_Bars_001_normal (Failed)
        243 - toktx-cmp-astc_mipmap_ldr_cubemap_6x6 (Failed)
        244 - toktx-cmp-astc_ldr_cubemap_6x6 (Failed)
        245 - toktx-cmp-astc_mipmap_ldr_6x6_posx (Failed)
        246 - toktx-cmp-astc_ldr_6x6_posx (Failed)
        247 - toktx-cmp-astc_mipmap_ldr_6x6_posz (Failed)
        248 - toktx-cmp-astc_mipmap_ldr_6x6_posy (Failed)
        249 - toktx-cmp-astc_mipmap_ldr_6x6_kodim17_fastest (Failed)
        250 - toktx-cmp-astc_mipmap_ldr_6x6_kodim17_fast (Failed)
        251 - toktx-cmp-astc_mipmap_ldr_6x6_kodim17_medium (Failed)
        252 - toktx-cmp-astc_mipmap_ldr_4x4_posx (Failed)
        253 - toktx-cmp-astc_mipmap_ldr_6x5_posx (Failed)
        254 - toktx-cmp-astc_mipmap_ldr_8x6_posx (Failed)
        255 - toktx-cmp-astc_mipmap_ldr_10x5_posx (Failed)
        256 - toktx-cmp-astc_mipmap_ldr_8x8_posx (Failed)
        257 - toktx-cmp-astc_mipmap_ldr_12x10_posx (Failed)
        258 - toktx-cmp-astc_mipmap_ldr_12x12_posx (Failed)
        259 - toktx-cmp-astc_ldr_4x4_FlightHelmet_baseColor (Failed)
        260 - toktx-cmp-astc_ldr_6x5_FlightHelmet_baseColor (Failed)
        261 - toktx-cmp-astc_ldr_8x6_FlightHelmet_baseColor (Failed)
        262 - toktx-cmp-astc_ldr_10x5_FlightHelmet_baseColor (Failed)
        263 - toktx-cmp-astc_ldr_8x8_FlightHelmet_baseColor (Failed)
        264 - toktx-cmp-astc_ldr_12x10_FlightHelmet_baseColor (Failed)
        265 - toktx-cmp-astc_ldr_12x12_FlightHelmet_baseColor (Failed)
        266 - toktx-cmp-astc_ldr_6x6_Iron_Bars_001_normal (Failed)
        268 - toktx-cmp-astc_ldr_6x6_arraytex_7 (Failed)
        269 - toktx-cmp-astc_ldr_6x6_arraytex_7_mipmap (Failed)
        270 - toktx-cmp-astc_ldr_6x6_3dtex_7 (Failed)
        330 - streamtestktxStreamTest.CanWriteKtx1AsKtx2ToCppStream (Failed)
        331 - streamtestktxStreamTest.CanWriteKtx2ToCppStream (Failed)

@Honeybunch
Copy link
Contributor

I also tried doing a build with a gcc distribution from msys2 and I got the same issue. Everything builds fine but there seem to be some possibly valid test failures. Still need to dig in with a debugger. Using the github actions on my fork to see if gcc 11.2.0 from choco gives the same test failures

@Honeybunch
Copy link
Contributor

Gotta run tests in release mode. gcc distro from w64devkit seems to be fine actually.
100% tests passed, 0 tests failed out of 330

@Honeybunch
Copy link
Contributor

Confirmed that with this working version of gcc 12 that we still need if (str->data.file->_flag & _IOWRT) I think this is because this distro of gcc is still targeting msvcrt. My current guess is that the choco dist of gcc is configured to link against ucrt instead; _IOWRT is only declared in the mingw headers if _UCRT is not defined.

@Honeybunch
Copy link
Contributor

YEP
The latest choco distro of mingw was changed to the ucrt version
image
Compared to:
image

@Honeybunch
Copy link
Contributor

Reproed this in the ucrt64 environment in msys2. I have a simple change that will get it building but we still get some test failures:

The following tests FAILED:
        147 - ktx2check-test-all (Failed)
        149 - ktx2check-test-stdin-read (Failed)
        150 - ktx2check-test-pipe-read (Failed)
        222 - toktx-cmp-luminance_reference_basis (Failed)
        227 - toktx-cmp-r_reference_basis (Failed)
        232 - toktx-cmp-swizzle_r_to_gb_convert_to_rgba_basis (Failed)
        ```

@Honeybunch
Copy link
Contributor

Oh I see why these are failing
image

@Honeybunch
Copy link
Contributor

Fixed the ucrt build by manually copying libstdc++-6.dll :(
image

@Honeybunch
Copy link
Contributor

GOT IT https://stackoverflow.com/questions/6404636/libstdc-6-dll-not-found
Setting mingw builds to use the -static-libgcc and -static-libstdc++ flags seems to be the fix. I'll get a PR up in a few

@Honeybunch Honeybunch mentioned this issue Oct 21, 2022
Merged
@MarkCallow
Copy link
Collaborator Author

Good grief! What an unholy mess from a change nothing to do with us. Thanks so much for delving into and fixing this.

I'll be away for the next 24 hours, leaving in a couple of minutes. I'll review the PR when I return.

MarkCallow pushed a commit that referenced this issue Oct 25, 2022
@Honeybunch
Fixing support for mingw toolchains that target the newer ucrt (#642)
0251377 
Fixes #641.
@Honeybunch Honeybunch mentioned this issue Oct 30, 2022
Closed
KaperD pushed a commit to KaperD/KTX-Software that referenced this issue Feb 21, 2024
@Honeybunch
Fixing support for mingw toolchains that target the newer ucrt (Khron…
cfb834e 
…osGroup#642)

Fixes KhronosGroup#641.
KaperD pushed a commit to KaperD/KTX-Software that referenced this issue Feb 22, 2024
@Honeybunch
Fixing support for mingw toolchains that target the newer ucrt (Khron…
cdb12cc 
…osGroup#642)

Fixes KhronosGroup#641.
KaperD pushed a commit to KaperD/KTX-Software that referenced this issue Feb 22, 2024
@Honeybunch
Fixing support for mingw toolchains that target the newer ucrt (Khron…
aa12555 
…osGroup#642)

Fixes KhronosGroup#641.
KaperD pushed a commit to KaperD/KTX-Software that referenced this issue Feb 22, 2024
@Honeybunch
Fixing support for mingw toolchains that target the newer ucrt (Khron…
d8348b5 
…osGroup#642)

Fixes KhronosGroup#641.
KaperD pushed a commit to KaperD/KTX-Software that referenced this issue Feb 22, 2024
@Honeybunch
Fixing support for mingw toolchains that target the newer ucrt (Khron…
435efd9 
…osGroup#642)

Fixes KhronosGroup#641.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixing support for mingw toolchains that target the newer ucrt Honeybunch/KTX-Software
2 participants
@MarkCallow @Honeybunch