Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot build with msvc x86 #38

Closed
izhengfan opened this issue Dec 26, 2019 · 25 comments
Closed

cannot build with msvc x86 #38

izhengfan opened this issue Dec 26, 2019 · 25 comments

Comments

@izhengfan
Copy link
Contributor

Under co/base/:

$ xmake f -a x86
checking for the Microsoft Visual Studio (x86) version ... 2019

$ xmake
[ 23%]: compiling.release co\impl\epoll.cc
[ 26%]: compiling.release co\impl\co_win.cpp
[ 29%]: compiling.release stack_trace\stack_trace_win.cpp
[ 33%]: compiling.release win\time.cpp
[ 39%]: compiling.release fast.cc
[  6%]: compiling.release fastring.cc
[  9%]: compiling.release hash\base64.cc
[ 16%]: compiling.release co\impl\co_unix.cc
[ 19%]: compiling.release json.cc
[ 42%]: compiling.release co\impl\scheduler.cc
[ 46%]: compiling.release unix\time.cc
[ 49%]: compiling.release win\os.cpp
[ 52%]: compiling.release str.cc
error: time.cpp
C:\Users\fzheng\source\repos\co\base\win/atomic.h(46): error C3861: '_InterlockedIncrement64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(62): error C3861: '_InterlockedDecrement64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(78): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(94): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(110): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(126): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(142): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(158): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(174): error C3861: '_InterlockedOr64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(190): error C3861: '_InterlockedAnd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(206): error C3861: '_InterlockedXor64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(222): error C3861: '_InterlockedOr64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(238): error C3861: '_InterlockedAnd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(254): error C3861: '_InterlockedXor64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(270): error C3861: '_InterlockedExchange64': identifier not found
@idealvin
Copy link
Owner

idealvin commented Dec 26, 2019

Under co/base/:

$ xmake f -a x86
checking for the Microsoft Visual Studio (x86) version ... 2019

$ xmake
[ 23%]: compiling.release co\impl\epoll.cc
[ 26%]: compiling.release co\impl\co_win.cpp
[ 29%]: compiling.release stack_trace\stack_trace_win.cpp
[ 33%]: compiling.release win\time.cpp
[ 39%]: compiling.release fast.cc
[  6%]: compiling.release fastring.cc
[  9%]: compiling.release hash\base64.cc
[ 16%]: compiling.release co\impl\co_unix.cc
[ 19%]: compiling.release json.cc
[ 42%]: compiling.release co\impl\scheduler.cc
[ 46%]: compiling.release unix\time.cc
[ 49%]: compiling.release win\os.cpp
[ 52%]: compiling.release str.cc
error: time.cpp
C:\Users\fzheng\source\repos\co\base\win/atomic.h(46): error C3861: '_InterlockedIncrement64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(62): error C3861: '_InterlockedDecrement64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(78): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(94): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(110): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(126): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(142): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(158): error C3861: '_InterlockedExchangeAdd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(174): error C3861: '_InterlockedOr64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(190): error C3861: '_InterlockedAnd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(206): error C3861: '_InterlockedXor64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(222): error C3861: '_InterlockedOr64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(238): error C3861: '_InterlockedAnd64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(254): error C3861: '_InterlockedXor64': identifier not found
C:\Users\fzheng\source\repos\co\base\win/atomic.h(270): error C3861: '_InterlockedExchange64': identifier not found

I have tried it on windows 10 with vs2015. Everything is ok.

Maybe you could try to add an undefine in xmake.lua as below:

    if is_plat("windows") then
        add_cxflags("-Ox", "-fp:fast", "-EHsc")
        add_files("**.cpp")
        if is_arch("x64") then
            add_files("co/context/context_x64.asm")
        else
            add_files("co/context/context_x86.asm")
            add_undefines("_WIN64")
        end
    end

@idealvin
Copy link
Owner

idealvin commented Dec 26, 2019

As I can know, on 32 bit platform, _Interlocked* are declared in the header file Windows.h.
I already add a detection in base/atomic.h

#ifdef _WIN32
#pragma warning (disable:4800)

#include <intrin.h>

#ifndef _WIN64
#define WIN32_LEAN_AND_MEAN
#include <Windows.h>
#endif

You may simply add add_undefines("_WIN64") in xmake.lua and try building again.

@izhengfan
Copy link
Contributor Author

Hi, thank you for your feedback. This is strange, I tried your solution but still got the same error.

Meanwhile, I check Windows.h in my machine, but cannot find _InterlockedIncrement64 there. Could it be that we have different versions of Windows SDK? Mine is 10.0.18362.0.

However, I check MSDN and it seems these _Interlocked*64 are not available on x86:

image

@idealvin
Copy link
Owner

I have found it in winnt.h:

LONGLONG
FORCEINLINE
_InterlockedIncrement64 (
    _Inout_ _Interlocked_operand_ LONGLONG volatile *Addend
    )
{
    LONGLONG Old;

    do {
        Old = *Addend;
    } while (InterlockedCompareExchange64(Addend,
                                          Old + 1,
                                          Old) != Old);

    return Old + 1;
}

@izhengfan
Copy link
Contributor Author

Now I find it in winnt.h too. Only that it is put under macro like #ifdef _AMD64_ , #if defined(_M_AMD64), so is inactive in Win32 mode.

@izhengfan
Copy link
Contributor Author

Anyway, would it be possible to disable these *64 api in x86 mode? After all according to msdn these should not work for x86 architecture.

@idealvin
Copy link
Owner

Should also work on x86? May you have a test on it?

@izhengfan
Copy link
Contributor Author

Not work here 😂 Maybe we need more guys or more machines to test if this is a real issue.

@izhengfan
Copy link
Contributor Author

I got another computer and freshly installed visual studio 2019 and xmake. Still x86 cannot be built, neither with 2015 build toolset nor 2019 build toolset. I guess this is an actual problem.

@dasuren
Copy link

dasuren commented Dec 26, 2019

I have the same issue when compiling in visual studio 2019 for x86.
report error error C3861: '_InterlockedExchangeAdd64': identifier not found
in the following function
inline long long atomic_fetch_inc(void* p, I<8>) {
return _InterlockedExchangeAdd64((long long*)p, 1);
}
but when I use "F12" to check the defination, visual studio 2019 open a file as follows:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.22.27905\include\intrin0.h
with following definations:
__MACHINEARM_ARM64_X64(__int64 _InterlockedExchangeAdd64(__int64 volatile * _Addend, __int64 _Value))

#define __MACHINEARM_ARM64_X64 __MACHINEZ

#define __MACHINEZ(X) /* NOTHING */

I guess izhengfan is right that x86 may not support it under certain versions, and they put /* NOTHING */ in their defination

@idealvin
Copy link
Owner

@izhengfan @dasuren
Could you look at here? Try to undef _MANAGED and build again?

@waruqi
Copy link
Collaborator

waruqi commented Dec 27, 2019

@idealvin Need to add declaration through pragma intrinsic first. msvc will use it as a built-in function without links.

__int64 _InterlockedExchangeAdd64(__int64 volatile* Destination, __int64 Value);
#pragma intrinsic(_InterlockedExchangeAdd64)

You can see https://github.com/tboox/tbox/blob/master/src/tbox/platform/windows/atomic64.h

@izhengfan
Copy link
Contributor Author

@idealvin undef _MANAGED does not work.

According to @waruqi 's comments, I added the lines below:

diff --git a/base/win/atomic.h b/base/win/atomic.h
index d53e146..733832e 100644
--- a/base/win/atomic.h
+++ b/base/win/atomic.h
@@ -8,6 +8,20 @@
 #ifndef _WIN64
 #define WIN32_LEAN_AND_MEAN
 #include <Windows.h>
+__int64 _InterlockedIncrement64(__int64 volatile* Dest);
+#pragma intrinsic(_InterlockedIncrement64)
+__int64 _InterlockedDecrement64(__int64 volatile* Dest);
+#pragma intrinsic(_InterlockedDecrement64)
+__int64 _InterlockedExchangeAdd64(__int64 volatile* Dest, __int64 value);
+#pragma intrinsic(_InterlockedExchangeAdd64)
+__int64 _InterlockedOr64(__int64 volatile* Dest, __int64 value);
+#pragma intrinsic(_InterlockedOr64)
+__int64 _InterlockedAnd64(__int64 volatile* Dest, __int64 value);
+#pragma intrinsic(_InterlockedAnd64)
+__int64 _InterlockedXor64(__int64 volatile* Dest, __int64 value);
+#pragma intrinsic(_InterlockedXor64)
+__int64 _InterlockedExchange64(__int64 volatile* Dest, __int64 value);
+#pragma intrinsic(_InterlockedExchange64)
 #endif

Now building of base, rpcgen and test pass, but unitest has link error:

$ xmake
[100%]: linking.release unitest.exe
error: atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedIncrement64(__int64 volatile *)" (?_InterlockedIncrement64@@YA_JPC_J@Z) referenced in function "__int64 __cdecl atomic_inc<__int64>(__int64 *)" (??$atomic_inc@_J@@YA_JPA_J@Z)
atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedDecrement64(__int64 volatile *)" (?_InterlockedDecrement64@@YA_JPC_J@Z) referenced in function "__int64 __cdecl atomic_dec<__int64>(__int64 *)" (??$atomic_dec@_J@@YA_JPA_J@Z)
atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedExchangeAdd64(__int64 volatile *,__int64)" (?_InterlockedExchangeAdd64@@YA_JPC_J_J@Z) referenced in function "__int64 __cdecl atomic_add<__int64,int>(__int64 *,int)" (??$atomic_add@_JH@@YA_JPA_JH@Z)
atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedOr64(__int64 volatile *,__int64)" (?_InterlockedOr64@@YA_JPC_J_J@Z) referenced in function "__int64 __cdecl atomic_fetch_or<__int64,int>(__int64 *,int)" (??$atomic_fetch_or@_JH@@YA_JPA_JH@Z)
atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedAnd64(__int64 volatile *,__int64)" (?_InterlockedAnd64@@YA_JPC_J_J@Z) referenced in function "__int64 __cdecl atomic_and<__int64,int>(__int64 *,int)" (??$atomic_and@_JH@@YA_JPA_JH@Z)
atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedXor64(__int64 volatile *,__int64)" (?_InterlockedXor64@@YA_JPC_J_J@Z) referenced in function "__int64 __cdecl atomic_fetch_xor<__int64,int>(__int64 *,int)" (??$atomic_fetch_xor@_JH@@YA_JPA_JH@Z)
atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedExchange64(__int64 volatile *,__int64)" (?_InterlockedExchange64@@YA_JPC_J_J@Z) referenced in function "void __cdecl atomic_reset<__int64>(__int64 *)" (??$atomic_reset@_J@@YAXPA_J@Z)
..\\..\\build\\unitest.exe : fatal error LNK1120: 7 unresolved externals

@waruqi
Copy link
Collaborator

waruqi commented Dec 27, 2019

atomic_test.cc.obj : error LNK2019: unresolved external symbol "__int64 __cdecl _InterlockedDecrement64(__int64 volatile *)" (?_InterlockedDecrement64@@YA_JPC_J@Z) referenced in function "__int64 __cdecl atomic_dec<__int64>(__int64 *)" (??$atomic_dec@_J@@YA_JPA_J@Z)

Try adding extern "C" { }

#ifdef __cplusplus
extern "C" {
#endif 

__int64 _InterlockedExchangeAdd64(__int64 volatile* Destination, __int64 Value);
#pragma intrinsic(_InterlockedExchangeAdd64)

#ifdef __cplusplus
}
#endif 

@izhengfan
Copy link
Contributor Author

@waruqi
Copy link
Collaborator

waruqi commented Dec 27, 2019

Because only _InterlockedCompareExchange64 exists for x86/msvc, so we need wrap other atomic64 apis using _InterlockedCompareExchange64.

@izhengfan
Copy link
Contributor Author

Sorry I still do not understand how to do it, could you directly fix the code in this repo?

@dasuren
Copy link

dasuren commented Jan 3, 2020

Hi All,
I am trying to use a dummy method to fix this issue as follows:

 #ifdef _WIN64
 namespace xx
 {
 inline long long atomic_swap(void* p, long long v, I<8>) {
 	return _InterlockedExchange64((long long*)p, v);
 }
 ...
 }
 #end

I moved all the 64 bits related functions to another block with #ifdef _WIN64.
In this case, all the 64bit version of code will be compiled on x64 compiler only.

For 64 bit,it compiles fine and pass all the unit test.
For 32 bits, base lib compiles fine but failed when compiling the unitest. with following error

C:\Users\andrewzhao\Documents\projects\co\base\win/atomic.h(383): note: while trying to match the argument list '(T *, type, it)'
with
[
T=int64
]

which could be caused by the test cases like follows:
EXPECT_EQ(atomic_inc(&i64), 1);

where a _int64 parameter are passed to the 32bit application, where related code are skipped.

Then I tried to replace _InterlockedExchange64 with some function which can be used under 32 bits and accept _int64 as parameter. However, it seems all the _int64 related functions are defined with _int64 parameter are end with **64.

I remove all the 64 bit test case under x86 compiler, and now it pass all the test cased.

So I have two questions: 1. do we need to use and test _int64 as input for x86 application?
2. if we do , any suggestion on how to solve this test issue?

@izhengfan
Copy link
Contributor Author

@dasuren I think this is a workaround, but not a good enough solution. For 32bit application, it is still desirable to have 64bit atomic operation.

@waruqi
Copy link
Collaborator

waruqi commented Jan 4, 2020

Need use _InterlockedCompareExchange64 to wrap other atomic64 apis for x86, because only _InterlockedCompareExchange64 exists for x86/msvc.

for example:

extern "C" {
__int64 _InterlockedCompareExchange64(__int64 volatile* Destination, __int64 Exchange, __int64 Comperand);
#pragma intrinsic(_InterlockedCompareExchange64)
}

static inline __int64 _InterlockedIncrement64 (__int64 volatile *Addend)
{
    __int64 Old;
    do {
        Old = *Addend;
    } while (_InterlockedCompareExchange64(Addend, Old + 1, Old) != Old);
    return Old + 1;
}

@dasuren
Copy link

dasuren commented Jan 4, 2020

Thanks Waruqi for your suggestion and help. I did some research on "#pragma intrinsic" and FORCEINLINE. but still got confused by code where three parameter "declared" but only one is defined.And visual studio think it is legal....Guess I need to go back to the text books...wow, feels like I am using a fake C++ in the past several year.. :)
Thanks again for that.

@izhengfan
Copy link
Contributor Author

izhengfan commented Jan 4, 2020

Thanks to @waruqi 's instructions, I now successful fix this problem, like this:

diff --git a/base/win/atomic.h b/base/win/atomic.h
index d53e146..a8e931f 100644
--- a/base/win/atomic.h
+++ b/base/win/atomic.h
@@ -8,6 +8,17 @@
 #ifndef _WIN64
 #define WIN32_LEAN_AND_MEAN
 #include <Windows.h>
+extern "C" {
+__int64 _InterlockedCompareExchange64(__int64 volatile* Destination, __int64 Exchange, __int64 Comperand);
+#pragma intrinsic(_InterlockedCompareExchange64)
+}
+#define _InterlockedIncrement64    InterlockedIncrement64
+#define _InterlockedDecrement64    InterlockedDecrement64
+#define _InterlockedExchangeAdd64  InterlockedExchangeAdd64
+#define _InterlockedOr64           InterlockedOr64
+#define _InterlockedAnd64          InterlockedAnd64
+#define _InterlockedXor64          InterlockedXor64
+#define _InterlockedExchange64     InterlockedExchange64
 #endif

 namespace xx {

Also, I notice that although x86 MSVC does not provide API like _InterlockedIncrement64, it does provide a wrapper InterlockedIncrement64 in winnt.h header, whose implementation is just like what waruqi has mentioned:

//@file winnt.h
LONGLONG
FORCEINLINE
_InlineInterlockedIncrement64 (
    _Inout_ _Interlocked_operand_ LONGLONG volatile *Addend
    )
{
    LONGLONG Old;

    do {
        Old = *Addend;
    } while (InterlockedCompareExchange64(Addend,
                                          Old + 1,
                                          Old) != Old);

    return Old + 1;
}

#define InterlockedIncrement64 _InlineInterlockedIncrement64

@idealvin
Copy link
Owner

idealvin commented Jan 6, 2020

@izhengfan
I have commented out these lines:

//extern "C" {
//__int64 _InterlockedCompareExchange64(__int64 volatile* Destination, __int64 Exchange, __int64 Comperand);
//#pragma intrinsic(_InterlockedCompareExchange64)
//}

Does it also work on your platform?

@izhengfan
Copy link
Contributor Author

@idealvin Yes, it still works. I guess win32 SDK has handled it well, so we don't need these lines.

@idealvin
Copy link
Owner

idealvin commented Jan 6, 2020

@idealvin Yes, it still works. I guess win32 SDK has handled it well, so we don't need these lines.

👌 Thanks for your work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants