Incorrect return value type used for Wow64EnableWow64FsRedirection #4681

Closed
wixbot opened this Issue Feb 20, 2015 · 5 comments

Projects

None yet

2 participants

@wixbot
Collaborator
wixbot commented Feb 20, 2015

src/libs/dutil/xmlutil.cpp: pfnEnableWow64 = (BOOL (WINAPI *)(BOOLEAN))::GetProcAddress(hKernel32, "Wow64EnableWow64FsRedirection");

The Wow64EnableWow64FsRedirection function (called by XmlCreateDocument) is declared as returning BOOL (32 bit). However it actually returns BOOLEAN (8 bit).
This means the upper 24 bits may be filled with garbage, which can lead to interpreting a return value of FALSE as TRUE. The effect of this would be termination without any error message (via ::ExitProcess(1); in the same function).

Most of the time this error will not be noticeable in practice because due to the implementation details of this function the "garbage" is determined by the last error value. The most common error code is ERROR_CALL_NOT_IMPLEMENTED (on 32-bit versions of Windows) with a value of 120, so the upper 24 bits happen to be zero; a rarer error code above 255 would exhibit this problem however.

Originally opened by thfabba

@wixbot
Collaborator
wixbot commented Feb 20, 2015

Fix:

diff --git a/src/libs/dutil/xmlutil.cpp b/src/libs/dutil/xmlutil.cpp
index d5e9c13..67a072d 100644
--- a/src/libs/dutil/xmlutil.cpp
+++ b/src/libs/dutil/xmlutil.cpp
@@ -130,7 +130,7 @@ extern "C" HRESULT DAPI XmlCreateDocument(
 {
     HRESULT hr = S_OK;
     BOOL (WINAPI *pfnDisableWow64)(__out PVOID* ) = NULL;
-    BOOL (WINAPI *pfnEnableWow64)(__in BOOLEAN ) = NULL;
+    BOOLEAN (WINAPI *pfnEnableWow64)(__in BOOLEAN ) = NULL;
     BOOL (WINAPI *pfnRevertWow64)(__in PVOID ) = NULL;
     BOOL fWow64Available = FALSE;
     void *pvWow64State = NULL;
@@ -147,7 +147,7 @@ extern "C" HRESULT DAPI XmlCreateDocument(
     if (NULL != GetProcAddress(hKernel32, "IsWow64Process"))
     {
         pfnDisableWow64 = (BOOL (WINAPI *)(PVOID *))::GetProcAddress(hKernel32, "Wow64DisableWow64FsRedirection");
-        pfnEnableWow64 = (BOOL (WINAPI *)(BOOLEAN))::GetProcAddress(hKernel32, "Wow64EnableWow64FsRedirection");
+        pfnEnableWow64 = (BOOLEAN (WINAPI *)(BOOLEAN))::GetProcAddress(hKernel32, "Wow64EnableWow64FsRedirection");
         pfnRevertWow64 = (BOOL (WINAPI *)(PVOID))::GetProcAddress(hKernel32, "Wow64RevertWow64FsRedirection");

         fWow64Available = pfnDisableWow64 && pfnEnableWow64 && pfnRevertWow64;

Originally posted by thfabba

@wixbot
Collaborator
wixbot commented Mar 3, 2015

AssignedTo set to bobarnson
Release changed from v3.x to v3.10

@wixbot
Collaborator
wixbot commented Mar 4, 2015

wixtoolset/wix3#218

Originally posted by barnson
Area set to extensions

@wixbot
Collaborator
wixbot commented Mar 4, 2015

#wix4

Originally posted by barnson

@wixbot
Collaborator
wixbot commented Mar 4, 2015

Originally changed by barnson
Resolution set to fixed
Status changed from Open to Resolved

@barnson barnson was assigned by wixbot Dec 20, 2015
@wixbot wixbot added this to the v3.10 milestone Dec 20, 2015
@wixbot wixbot closed this Dec 20, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment