Support of UNICODE macro on Windows Visual Studio Projects

[jace-bevo]

We are using S3Client class to get and put object from Windows client. We encounter the following link error from Visual Studio 2015 on Windows 10:
6>Cloud.obj : error LNK2019: unresolved external symbol "public: virtual class Aws::Utils::Outcome<class Aws::S3::Model::GetObjectResult,class Aws::Client::AWSError > __cdecl Aws::S3::S3Client::GetObjectW(class Aws::S3::Model::GetObjectRequest const &)const " (?GetObjectW@S3Client@S3@Aws@@ueba?AV?$Outcome@VGetObjectResult@Model@S3@Aws@@v?$AWSError@W4S3Errors@S3@Aws@@@client@4@@utils@3@AEBVGetObjectRequest@Model@23@@z) referenced in function "private: int __cdecl CloudAuthenticator::RetrieveKey(class std::basic_string<char,struct std::char_traits,class std::allocator > const &)" (?RetrieveKey@CloudAuthenticator@FMS@@AEAAHAEBV?$basic_string@DU?$char_traits@D@std@@v?$allocator@D@2@@std@@@z)
As you can see that there is only Aws::S3::S3Client::GetObject, not GetObjectW API. This is because all our projects enable 'Use Unicode Character Set' of 'Character Set' property, therefore 'UNICODE' is defined and Visual Studio's header file in \Program Files(x86)\Windows Kit\8.1\incude\um\wingdi.h has the following macro

#ifdef UNICODE
#define GetObject GetObjectW
#else
#define GetObject GetObjectA
#endif // !UNICODE

This causes the preprocessor to map 'GetObject' into 'GetObjectW'.
However, none of the VS project files generated by Cmake for AWS Cpp SDK turns on so the resulting libraries do not have the correct symbol.
Is there a way we can make Cmake generate Unicode compatible project files?

[singku]

It's a name collision with the Windows function. You can wrap up GetObject in S3Client with a new function before including windows headers. Then call the new function instead of GetObject.

[jace-bevo]

I worked around it with the following code:
std::unique_ptr< Aws::S3::S3Client > s3c(new Aws::S3::S3Client( config ));
auto getObjectOutcome = s3c.get()->GetObject( getObjectRequest );
instead of
Aws::S3::S3Client s3c( config );
auto getObjectOutcome = s3c.GetObject( getObjectRequest );
I am still not clear why this fixes the problem but it does. Maybe the preprocessor does not make the name mapping change if it is referenced from an object pointer instead of an object instance?

Anyway the real fix is to add support of 'UNICODE' in Windows for AWS Cpp SDK. The GetObject API is already published so it cannot be renamed and most likely there are other name collisions, too.

[JonathanHenson]

the support plan for unicode is to never use it anywhere unless it is utf-8. The issue here is not related to unicode at all. The issue is a name collision caused by windows' way of handling unicode. And in that case you'll need to undef the define. Our contract is that users pass const char* or std::string. We provide utilities to convert between wchar* and char* in Aws::Utils::StringUtils. wchar and cross-platform do not mix.

…

Sent from my iPhone

On Aug 2, 2017, at 3:34 PM, jace-bevo ***@***.***> wrote: I worked around it with the following code: std::unique_ptr< Aws::S3::S3Client > s3c(new Aws::S3::S3Client( config )); auto getObjectOutcome = s3c.get()->GetObject( getObjectRequest ); instead of Aws::S3::S3Client s3c( config ); auto getObjectOutcome = s3c.GetObject( getObjectRequest ); I am still not clear why this fixes the problem but it does. Maybe the preprocessor does not make the name mapping change if it is referenced from an object pointer instead of an object instance? Anyway the real fix is to add support of 'UNICODE' in Windows for AWS Cpp SDK. The GetObject API is already published so it cannot be renamed and most likely there are other name collisions, too. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[jace-bevo]

I understand that the user is responsible for passing std::string to AWS. However, what I meant for 'UNICODE' support is the macro definition in VS project, not 'Unicode' functionality support in AWS Cpp SDK.
I have tried to #undef UNICODE in our code but it causes a whole slew of other problems. We, too, have similar issue in our code where we have API as 'GetUserName' and 'GetPassword' that collides with VS libraries. But since they are in our code space the resulting libraries all have the corresponding symbols so we do not have name collision issue.
What I am asking for is the support of letting users 'turn on' UNICODE macro definition in the VS project files generated by Cmake to avoid symbol mismatch problems if user's existing project support 'Unicode' and UNICODE is defined in their project.

[singku]

When you said turn on, did you mean functions supplied by SDK also has the W/A suffix? So as to avoid name collision? Otherwise I don't see why that would help.

[jace-bevo]

What I suggest is that the SDK lets users build the project with a flag, say -DUNICODE_CHARSET_PROPERTY, when generating project files for Visual Studio with cmake so that the property of the vcxproj sets the property <Character Set>Unicode</Character Set>.
(The default behavior does not this property.)

[singku]

Would that help you solve this problem and how?

[jace-bevo]

See my reply to your first comment. It is an ugly hack but at least I do not unresolved symbol errors from the linker and the binary is created fine.
We should not expect users whose Windows project is 'Unicode' enabled to do this in the code to work around the issue since we do not how many similar cases there will be.

[singku]

You mean settling up the vs project file to Unicode without definition of UNICODE?

[jace-bevo]

No, I mean changing my code from

Aws::S3::S3Client s3c( config );
auto getObjectOutcome = s3c.GetObject( getObjectRequest );

to

std::unique_ptr< Aws::S3::S3Client > s3c(new Aws::S3::S3Client( config ));
auto getObjectOutcome = s3c.get()->GetObject( getObjectRequest );

Since this involve changing users' source code manually everything they encounter the issue, it is not practical.

[singku]

I understand your work around here. My point is as long as there is macro definition of UNICODE and you include these Windows headers. There will be name collision anyway. No matter we default it to unicode or not.

[singku]

@jace-bevo That's why I was asking is there away to default unicode property in VS project file without macro definition of UNICODE.

[jace-bevo]

If cmaje can generate the VS project with the UNICODE macro defined, hopefully it will cause the code to be compiled with the GetObject API mapped into GetObjectW by some Window specific headers file included by the SDK.
If you use VS to open the soluiton or project files generated for AWS Cpp SDK for building, and click on other external references of S3Client::GetObject, what do you get? Is it the actual definition of the API or macro dedinition from a VS header file like the one in the very top of this thread?

[jace-bevo]

The VS header file that redefine GetObject is not directly included in our code. It is indirectly included by other header files we use that I do not even know which.

[singku]

It's the original definition in my project. As I said, unless you find a way to bypass the macro expand of GetObject to GetObjectW/A for your S3 function call, define UNICODE for you will do no help to solve this problem.

The reason is: macro UNICODE is incorporated either by vs setting or by CMake (as you expected), so any appearance of GetObject after the windows header including, here is wingdi.h (it doesn't matter directly or indirectly included) will be replaced with GetObjectW/A. We don't have s3Client::GetObjectW/A definitions in SDK, so it will not be compiled. The work around you have done is to cheat preprocessor; other ways incuding undef UNICODE before call these functions as Jonathan said and redefine it after call; Or you can wrap it up as I said;

[jace-bevo]

When you set the <Character Set> property in the project, Visual Studio implicitly defines that macro and it affects all other include files. It should be easy to reproduce the problem I see.
Just create a simple C++ project in VS, set the character set to 'Unicode' in the project's General property panel, add some code that calls S3Client.GetObject like I have above. Build it with the necessary AWS libraries and see what you get.

[singku]

I did that for my test project and it worked fine.

[diablodale]

I may have a workaround for you. I had collisions with both GetMessage() and GetObject(). I also compile with VC++ Unicode enabled. My workaround is to #undef the Windows macros before loading the AWS SDK. For example:

#define NOMINMAX           // required for AWS SDK to compile on Windows issue#641
#include <windows.h>
#include <atlbase.h>
#undef GetMessage          // workaround for AWSError method GetMessage(), and ATL conflict
#undef GetObject           // workaround for Aws::S3::S3Client::GetObject
#include <aws/s3/S3Client.h>
#include <aws/s3/model/PutObjectRequest.h>
#include <aws/transfer/TransferManager.h>

This is related to #402

[singku]

@jace-bevo It's been a while after last reply, I think the work around should work fine. I am going to close this issue, If you have any further problem, please reopen it.