Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

`*windows-default-manifest` package issues #454

Closed
vszakats opened this issue Feb 19, 2016 · 41 comments

Comments

@vszakats
Copy link
Contributor

commented Feb 19, 2016

Hi,

Since this patch, when using MSYS2/mingw-w64, a default Windows manifest is unconditionally added to any linked application if that default resource object file is found.

This causes issues:

  • If a user's build process already sets its own manifest, the resulting binary will be corrupted. (with varying consequences — I got an UPX error message (upx: test.exe: CantPackException: superfluous data between sections), which could be worked around by a strip test.exe command, but which also stripped valid parts off of the internal manifest.)
  • User may want to override the default manifest for various reasons, so a forced system default may not be desired at all times.
  • User cannot override/drop the default manifest using a build-time option. (AFAICS)
  • User may uninstall the related package, but that fails if it was installed via a package group (?) (as seems to be the case f.e. in AppVeyor's default MSYS2 installation), which also appears to be the natural and recommended way of installing mingw-w64. Uninstall failure here:
$ pacman -Suy
[...]
$ pacman -Rs mingw-w64-x86_64-windows-default-manifest
checking dependencies...
error: failed to prepare transaction (could not satisfy dependencies)
:: mingw-w64-x86_64-gcc: removing mingw-w64-x86_64-windows-default-manifest breaks dependency 'mingw-w64-x86_64-windows-default-manifest'
  • Uninstalling these components also means to alter the system state, which may badly interact with other projects that might actually find a default manifest useful (or are built to rely on it):
    • mingw-w64-i686-windows-default-manifest
    • mingw-w64-x86_64-windows-default-manifest
    • mingw-w64-cross-windows-default-manifest (possibly)
    • windows-default-manifest
  • Manually deleting related files is very hacky and also difficult to maintain. It may also not work on systems where the build process has no write access to system files (i.e. Unix cross-builds):
    • /mingw32/i686-w64-mingw32/lib/default-manifest.o
    • /mingw64/x86_64-w64-mingw32/lib/default-manifest.o
    • /usr/lib/default-manifest.o
  • Separately downloaded (non-MSYS2) mingw-w64 builds don't have this issue, because they are missing the default manifest component.

Everything considered, it might be a satisfactory solution to detach these *windows-default-manifest packages from the mingw-w64-{x86_64,i686}-toolchain package group (or whichever groups they are part of), and let users decide to install it or not. (I'm quite in the dark with pacman packaging details, and couldn't so far find more information about it.)

Overall, the goal would be to keep using existing, custom manifests and disable automatic linkage of any default ones. What would be the best/recommended way to achieve this goal?

[Sorry if this is the wrong place to report this. I'm not exactly sure if this is an MSYS2/Cygwin/mingw-w64 packaging issue, an installation issue (?), or a mingw-w64 toolchain issue. For sure it also involves a linker bug though.]

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 19, 2016

Steps to reproduce a corrupt .exe:

Requires MSYS2 with mingw-w64 newer than ~2014 October (possibly also works with built-in mingw 4.9.2 — not tested).

Tested with: mingw-w64 5.3.0-2

test.c: (dllexport is important, so that a relocation table gets generated after the resource section.)

#include <stdio.h>

__attribute__((dllexport)) int main(void)
{
   printf("hello\n");
   return 0;
}

win.rc:

1 24 /* RT_MANIFEST */
BEGIN
   "<?xml version=""1.0"" encoding=""UTF-8"" standalone=""yes""?>"
   "<assembly xmlns=""urn:schemas-microsoft-com:asm.v1"" manifestVersion=""1.0"">"
   "</assembly>"
END

making sure to have the default manifest package installed:

pacman -S mingw-w64-i686-windows-default-manifest

build:

$ windres -F pe-i386 win.rc -o win.o
$ gcc -m32 test.c -c
$ gcc -m32 test.o win.o -o test.exe -s -v

compress (using UPX 3.91):

$ upx test.exe
  → upx: test.exe: CantPackException: superfluous data between sections
$ strip -g test.exe
  → orphan resource data stripped
$ upx test.exe
  → OK

It may well indicate an UPX problem, too — one that's surfacing with such an invalid binary.

Nevertheless, the duplicate, orphan, default resource is what causes the chain of events.

@mingwandroid mingwandroid self-assigned this Feb 19, 2016

@mingwandroid

This comment has been minimized.

Copy link
Member

commented Feb 20, 2016

This is one of those bugs that requires significant effort to even know which project it belongs to! We can definitely rule out Cygwin (or the msys2-runtime fork) though, it's between UPX and MinGW-w64's GCC/binutils.

I'd like to bring it to Kai Teitz's attention but I feel we should spend more time on it first, however I am far too busy. You've done a lot of work on it already, and that's great, so I'm tempted to offer to try to help you to help yourself, if possible ... So, is there anything at all that you need to know about MSYS2 that would allow you further your own investigations? I'm thinking for example things about how to build debug versions of packages and advice about how to debug them in MSYS2? Ask here or jump into IRC (#msys2 on OFTC) if we can be of any use in that regard.

I'm thinking the bug surface area is just too large and you seem capable of zero-ing in effectively, so there's probably no one better for the job given your head is exactly in this space.

I was going to suggest bringing it up on GCCs bugzilla or sending a mail to MinGW-w64's mailing list, but I think it's a bit to vague at present.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

Thanks @mingwandroid! I'm going to register to GCC Bugzilla and follow-up on it.

@mingwandroid

This comment has been minimized.

Copy link
Member

commented Feb 20, 2016

Ok, please keep this issue updated, it certainly looks like a complicated one. UPX always does complicate things.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

I'll definitely update this.

Till it gets sorted out throughout the toolchain/UPX (both would take quite some time I reckon), do you see any option to work this around by uninstalling the default manifest packages?

I understand they're useful for some apps and most certainly when building MSYS/Cygwin's own binaries, but for user apps in general, it's not always a blessing. ld is known to have issues with multiple Windows resource objects since possibly the very beginning.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

@mingwandroid Just read your updated message above. Many thanks for offering your help! I was busy filing the Bugzilla report, which is now finished. It contains some extra bits of information, but it's still difficult to tell which component will be the best candidate for a fix. Maybe multiple.

For now, the issue could possibly be avoided from the MSYS2 side by uninstalling the *windows-default-manifest MSYS2 packages, which at the moment trigger the rest of the problems. This is not an ultimate solution, only a local workaround of course.

I'm new to pacman package management, so certain things are unclear.

  1. Based on the pacman uninstall error message, it seems that the default-manifest package is a hard dependency of mingw-w64-x86_64-gcc:
    :: mingw-w64-x86_64-gcc: removing mingw-w64-x86_64-windows-default-manifest breaks dependency 'mingw-w64-x86_64-windows-default-manifest'
    I understand though, that the default-manifest is technically an optional component, because mingw-w64 would work perfectly well without it. The only difference would be the lack of the manifest object file (default-manifest.o) that is causing all the problems. So, one option could be to make this package dependency a soft/optional one in MSYS2 and allow it to be uninstalled, without uninstalling mingw-w64-x86_64-gcc with it. Or, removing it from the mingw-w64-x86_64-gcc dependency list completely and let it exist as an independent package (this one has more far-reaching consequences).
  2. Another approach could be to install each mingw-w64-x86_64-gcc subcomponent one-by-one, and skip the default-manifest package. If this is feasible, how to find out what are the subcomponents of mingw-w64-x86_64-gcc?

Can you help assessing the two ideas above?

[ If none of these is possible, an ugly, but effective hack would be to manually delete all the offending objects — either before starting a build or after each MSYS2 update. ]

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

Package information here:

It means assumptions were correct — for some reason the default manifests are being shipped as an inseparable part of gcc. Above files also describe the rest of the included dependencies.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

Patches that enabled default manifests:

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 20, 2016

BTW, if anyone wonders, manifests — speaking about these default ones — serve the purpose to force certain Windows API calls (non-deprecated ones included, like VerifyVersionInfo()) to return the true OS version. Maybe they have other uses, too. AFAIU they should be added to the manifest when the application is known to be compatible with these newer OS iterations.

It would be interesting to know the reason behind the decision to apply this automatically to all built binaries.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 21, 2016

Workaround implemented for the time being:
vszakats/harbour-core@e912b3b

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2016

Wouldn't optdepends (instead of depends) for *windows-default-manifest in mingw-w64-{x86_64,i686}-gcc packages be a solution here?

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Mar 30, 2016

I'm closing this without solution or feedback.

@vszakats vszakats closed this Mar 30, 2016

@Alexpux Alexpux reopened this Mar 30, 2016

@Alexpux

This comment has been minimized.

Copy link
Member

commented Mar 30, 2016

Sorry I have no time to handle all issues. We need to think how to do it properly

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Mar 30, 2016

Thanks for jumping in, no problem to keep it open if there is an interest in this. Thanks!

vszakats added a commit to vszakats/harbour-core that referenced this issue Aug 8, 2016

2016-08-08 13:23 UTC+0200 Viktor Szakats (vszakats users.noreply.gith…
…ub.com)

  * package/mpkg_win_ci.sh
  * package/mpkg_win_dl.sh
    % do not bundle upx.exe with the Harbour package. Besides being
      another maintenance burden, due to some MSYS2/Cygwin decisions,
      binutils bugs and upx properties, by default, MSYS2-built
      binaries may not always be UPX-ed. The resolution is stalled
      for now and will probably take some years to get sorted out.
      Refs:
         msys2/MSYS2-packages#454
         https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69880
      Anyhow, UPX-ing binaries have some drawbacks because the image
      cannot be mmap-ed and/or loaded on demand and/or shared
      between processes. It also mangles the content so it's doesn't
      help transparency. So, unless the binary is executed through
      the local network from a network share, it's not very useful.
      Get latest version via MSYS2
         pacman -S upx
      or from:
         https://fossies.org/windows/misc/upx391w.zip
@wyldckat

This comment has been minimized.

Copy link

commented Nov 21, 2016

@vszakats Many thanks for the detailed steps and documentation you've taken so far! I'm having this issue with a project as well (auto-mentioned above), therefore the workaround is also very much appreciated.

To answer the question about why the manifesto is embedded by default, it's documented here: https://sourceforge.net/p/mingw-w64/wiki2/default_manifest/

It was rather unfortunate that it auto-links it in, but at least renaming/removing the file is a relatively simple way to workaround the issue.

In the meantime, I'll try to look into the patch that was provided at GCC's bugzilla. I have some experience in cross-compiling the whole GCC stack on Linux for Windows, so it should be relatively easy for me to apply the patch and test it on my existing development stack.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Nov 21, 2016

@wyldckat You're welcome and thanks for the link too. Looking forward for your bintools results.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Dec 12, 2016

UPX release 3.92 (released yesterday) implements a workaround where it allows to override the superfluous data between sections error by using the --force option. This resulted in a correctly working compressed .exe (tested with a single test case that failed to compress earlier).

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Jan 10, 2017

I'm closing this because the Issue on this level now has sufficient workarounds and the root cause got its own (still open) Issue on GCC bugzilla, with a proposed, but untested yet patch.

@Ede123

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2017

Just chiming in to "optimize" some searches to hopefully make this easier to debug for others.

While building libcdr and libvisio this very issue caused the build process to fail silently (i.e. no error message at all) while linking cdr2raw.exe and vsd2raw.exe respectively.

Only a non-zero return code of ld indicated something went wrong:
collect2.exe: error: ld returned 5 exit status

Edit: Also seems to be highly arbitrary. libcdr-0.1.2 failed to build, libcdr-0.1.3 worked (both with mingw64). libvisio-0.1.5 failed when built with mingw64 but succeeded in mingw32.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 21, 2017

Thank you @mati865!

I've rerun the test (for both 32 and 64-bit) and the problem appears to be fixed now: UPX 3.91 doesn't complain, and the binary doesn't contain stray bits of the default manifest anymore. (Done a negative test without a custom manifest and the default ones were linked as expected.)

vszakats added a commit to vszakats/harbour-core that referenced this issue Feb 21, 2017

2017-02-21 19:18 UTC Viktor Szakats (vszakats users.noreply.github.com)
  * package/mpkg_win_dl.sh
    % remove MSYS2/mingw default-manifest.o hack after proposed patch
      landed upstream fixing the issue in binutils:
        msys2/MINGW-packages@0e79e62
      Refs:
        msys2/MSYS2-packages#454
        https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69880
@Alexpux

This comment has been minimized.

Copy link
Member

commented Feb 22, 2017

@vszakats report about in on bugzilla

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2017

@Alexpux Reported my new test results there yesterday: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69880#c13

Thanks for merging!

@Ede123

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

Also fixes msys2/MINGW-packages#2190 after re-building gcc.

However the fix is probably not necessary anymore in the next release of binutils as there was an alternate fix in the meantime (untested),

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2017

While doing tests yesterday, I noticed that UPX 3.91 keeps returning the same error with certain large binaries even with no resources linked (and default resource neutralised). It means there are some other issues in current binutils. This was reproducible using non-MSYS2 binutils (2.27) build as well. (64-bit was tested). It will be interesting to repeat these once the alternate fix landed.

BTW, is there a dedicated tool to check/validate Windows .exe files? It'd be nice to use something less accidental than UPX for these tests :) [even though UPX is great at returning a yay/nay result. ]

@mati865

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

I have compiled biutils with https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=ec8f76882145c71bef81a9cadf0bf51ff9fa5b35 instead of proposed patch.
@vszakats your test is working:

$ upx test.exe
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2013
UPX 3.91        Markus Oberhumer, Laszlo Molnar & John Reiser   Sep 30th 2013

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
     19968 ->      8704   43.59%    win64/pe     test.exe

Packed 1 file.

If you want to try it:
mingw-w64-x86_64-binutils-2.27-3-any.pkg.tar.xz.txt (remove .txt, stupid github; version is lower than current binutils in repo so it will get replaced when you do update)

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2017

Thanks @mati865. I've rebuilt the of the large (16MB) binary with your build: The resource looks fine, the (unrelated) UPX message (upx: app.exe: CantPackException: superfluous data between sections) is there. With another (2MB) real-world binary both resource and UPX is fine.

It seems the new patch fixes the resource issue correctly and just as well as the first one.

@mati865

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

Thank you for testing @vszakats.
I'll open PR when I get 32 bit version building.

You should ask on mailing list or report it to bugzilla.

@Ede123

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

I have compiled biutils with https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=ec8f76882145c71bef81a9cadf0bf51ff9fa5b35 instead of proposed patch.

Great! Also fixes msys2/MINGW-packages#2190 (this time in fact without the need for rebuilding gcc).

mati865 added a commit to mati865/MINGW-packages that referenced this issue Feb 22, 2017

@mati865

This comment has been minimized.

Copy link
Contributor

commented Feb 22, 2017

Great, thanks for testing.

mati865 added a commit to mati865/MINGW-packages that referenced this issue Feb 22, 2017

Alexpux added a commit to msys2/MINGW-packages that referenced this issue Feb 24, 2017

@IlyaBizyaev

This comment has been minimized.

Copy link

commented Apr 26, 2017

Same problem: can't compress with UPX, superfluous data between sections. MSYS2 x64, latest packages.

@mati865

This comment has been minimized.

Copy link
Contributor

commented Apr 26, 2017

@IlyaBizyaev can you test UPX 3.93 (in repo there is 3.91 version)?

@IlyaBizyaev

This comment has been minimized.

Copy link

commented Apr 26, 2017

@mati865 Tested, same issue

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Apr 26, 2017

This has been confirmed earlier, but the superfluous data is unrelated to the manifests (the subject of this Issue). Likely a different binutils issue.

@andlabs

This comment has been minimized.

Copy link

commented Sep 2, 2018

How can I disable this default manifest from a build option? I am building a package for Go that allows writing GUI programs, and Go uses gcc for linking C modules. Since I'm not specifying any manifest files in the C modules (intentionally, to allow them to be overridden by users), and due to the way Go links stuff with gcc, the manifest is automatically added to my code (and then linked into code that uses my code). This is a problem because I need to use Common Controls v6, and the embedded manifest means Windows never even looks at any manifest file in the same directory as the executable.

Also how does this have no errors when merging two .a files, both specifying manifests? With the way Go builds packages that depend on C modules, two packages will each have their own embedded versions of the manifest. Is there no conflict because they are identical? Does one win out over the other?

@wyldckat

This comment has been minimized.

Copy link

commented Sep 2, 2018

@andlabs: If I remember correctly, this also affects Cygwin, not just MSYS2, as well as cross-compiling on other platforms (e.g. compiling Windows-binaries from Linux).
And AFAIK, the only guaranteed way to enforce that the default manifest is not used was mentioned in the first post above, namely to remove said object files.

As for the .a files, I'm not certain if that is supported. I know that adding additional .o files with manifests built into them for the final link is already supported, it was fixed sometime ago, if I remember correctly... see the comments above for Feb 22, 2017.

You may want to try and unpack the .a files back into .o files and then link those instead in the final build, to see if that solves the issue. If not possible, it's best that you create a small test package and report it at https://gcc.gnu.org/bugzilla - given that this is something that needs to be fixed upstream, since it affects all GCC/binutils features for building Windows-compatible binaries.

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Sep 2, 2018

Cygwin is most likely affected, that's where the idea of default manifest originates from. Can't confirm the cross-compiling case though — with mingw-w64 toolchains on macOS (via Homebrew) and Linux, the package responsible for installing the default manifest files does not exist.

@wyldckat

This comment has been minimized.

Copy link

commented Sep 2, 2018

@vszakats Many thanks for the reminder! Indeed, this was implemented on GCC for Cygwin and MSYS2 inherited it.

Following up on the mailing list thread you originally mentioned, there is this email that explains what should happened when more than one manifest file exists: https://gcc.gnu.org/ml/gcc-patches/2014-04/msg01382.html

Mmm... however, it was implemented directly into GCC upstream, so any bug reports for it should still be presented on its bug tracker.

@andlabs

This comment has been minimized.

Copy link

commented Sep 2, 2018

Er yeah, sorry I meant to say .o files, not .a files.

I'll probably just provide a package with a manifest that people can optionally use, assuming the merging facility does indeed work even with .o files that already have the default manifest. There are also other manifest generation packages (see the issue I linked this one to) that should work if these do. I'll have to try it and see. It's a shame I would have to tell users (other programmers) to remove these files in the worst case though, but oh well :/

@vszakats

This comment has been minimized.

Copy link
Contributor Author

commented Sep 2, 2018

@wyldckat You're right, the feature to pick up and use a default manifest if present on disk are implemented in the GCC toolchain indeed. (They are only present in MSYS2/Cygwin though.)

antoniou79 added a commit to antoniou79/scummvm that referenced this issue Aug 1, 2019

DISTS: Fix OpenGL renderer issue for builds with MSYS2/Mingw64 or MSY…
…S2/Mingw32

The issue pertains to MSYS2 adding a default manifest file (default-manifest.o) to the executable

The bug is for PC systems with GPU drivers that were not properly supported for Windows 10 systems, like
Intel HD Graphics series 1rst and 2nd generations. In those systems, launching a game in ScummVM (built with MSYS2/Mingw)
with the OpenGL renderer would cauase the game screen to be a white blank image, and various warnings would be output to the console:
eg.
"WARNING: GL ERROR: GL_INVALID_ENUM on glTexSubImage2D(0x0DE1, 0, 0, area.top, src.w, area.height(), _glFormat, _glType, src.gere.cpp:167)!"
This was due to MSYS2/Mingw builds trying to load the (poorly supported) GPU driver while advertising support for Windows 10 in their
embedded default Manifest file. Hence, the GPU driver dll (eg ig4icd64.dll) would be unloaded, causing the bug.
More information is available in the following links:
https://github.com/pal1000/save-legacy-intel-graphics
LWJGL/lwjgl#119
msys2/MSYS2-packages#454
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69880

bluegr added a commit to scummvm/scummvm that referenced this issue Aug 4, 2019

WIN32: Add a default application manifest
This fixes an OpenGL renderer issue for builds with MSYS2/Mingw64 or MSYS2/Mingw32

The issue pertains to MSYS2 adding a default manifest file (default-manifest.o) to the executable

The bug is for PC systems with GPU drivers that were not properly supported for Windows 10 
systems, like Intel HD Graphics series 1st and 2nd generations. In those systems, launching a 
game in ScummVM (built with MSYS2/Mingw) with the OpenGL renderer would cause the game 
screen to be a white blank image, and various warnings would be output to the console, eg.
"WARNING: GL ERROR: GL_INVALID_ENUM on glTexSubImage2D(0x0DE1, 0, 0, area.top, src.w, area.height(), _glFormat, _glType, src.gere.cpp:167)!"
This was due to MSYS2/Mingw builds trying to load the (poorly supported) GPU driver while advertising support for Windows 10 in their
embedded default Manifest file. Hence, the GPU driver DLL (eg ig4icd64.dll) would be unloaded, causing the bug.

More information is available in the following links:
https://github.com/pal1000/save-legacy-intel-graphics
LWJGL/lwjgl#119
msys2/MSYS2-packages#454
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=69880

Credits to sluicebox for the VS GenerateManifest flag
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.