Add System.Globalization.Native support for Windows #33704

safern · 2020-03-18T04:19:22Z

This enables building System.Globalization.Native on Windows (just building it on coreclr) since we shouldn't care about the windows native shim being built on the libraries side until the managed code is changed. I hope that by that time we have it linked in mono as well so that we can entirely stop building the shim in libraries at all.

This also enables loading icu.dll on Windows.

Contributes to: #826

cc: @jkotas @tarekgh

src/coreclr/src/dlls/mscoree/coreclr/CMakeLists.txt

src/libraries/Native/AnyOS/System.Globalization.Native/CMakeLists.txt

src/libraries/Native/AnyOS/System.Globalization.Native/configure.cmake

src/libraries/Native/AnyOS/System.Globalization.Native/pal_icushim.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_locale.h

src/libraries/Native/AnyOS/System.Globalization.Native/pal_calendarData.h

src/libraries/Native/AnyOS/System.Globalization.Native/pal_collation.c

jkotas · 2020-03-18T04:32:55Z

src/libraries/Native/AnyOS/System.Globalization.Native/pal_collation.c

@@ -403,8 +407,12 @@ static const UCollator* GetCollatorFromSortHandle(SortHandle* pSortHandle, int32
        pCollator = CloneCollatorWithOptions(pSortHandle->collatorsPerOption[0], options, pErr);
        UCollator* pNull = NULL;

+#ifdef TARGET_UNIX
        // we are not using the standard atomic_compare_exchange_strong to workaround bugs in clang 5.0 (https://bugs.llvm.org/show_bug.cgi?id=37457)


Do we still need this workaround? I believe we are not building with clang5 anymore.

Yeah. We now use clang9 so we can remove this workaround. I will fix it as part of this.

If I try to use atomic_compare_exchange_strong I get an error:

/home/runtime/src/libraries/Native/Unix/System.Globalization.Native/pal_collation.c:410:46: error: _Atomic cannot be applied to incomplete type 'UCollator' (aka 'struct UCollator')
if (!atomic_compare_exchange_strong((_Atomic(UCollator) *)&pSortHandle->collatorsPerOption[options], &pNull, pCollator))
/usr/include/unicode/ucol.h:58:8: note: forward declaration of 'struct UCollator'
struct UCollator;

It seems like the declaration of UCollator in the icu headers is incomplete.

maybe you need to ensure

#include "unicode/uloc.h"

is included?

It was included. I managed to do it with void* volatile*. I'll put an update soon.

It seems like in gcc atomic_compare_exchange_strong uses __atomic_compare_exchange macro which expects other parameter types, if I instead directly use __atomic_compare_exchange_n in gcc builds just fine. Will leave as is in order to have a green build at the moment.

/__w/1/s/src/libraries/Native/Unix/Common/pal_atomic.h:15:12: error: size mismatch in argument 2 of '__atomic_compare_exchange'
return atomic_compare_exchange_strong_explicit((_Atomic(void*)volatile*)dest, exchange, comperand, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST);

src/libraries/Native/AnyOS/System.Globalization.Native/pal_collation.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_icushim.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_icushim_internal.h

src/libraries/Native/AnyOS/System.Globalization.Native/pal_calendarData.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_icushim.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_idna.c

tarekgh · 2020-03-18T16:41:49Z

CC @janvorli

tarekgh · 2020-03-18T16:42:45Z

modulo the comments mentioned here, LGTM

src/libraries/Native/AnyOS/System.Globalization.Native/pal_collation.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_idna.c

src/libraries/Native/AnyOS/System.Globalization.Native/pal_icushim_internal.h

src/libraries/Native/AnyOS/System.Globalization.Native/pal_calendarData.c

src/coreclr/src/libraries-native/entrypoints.c

src/libraries/Native/AnyOS/Common/compiler.h

src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c

…sIcu

src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c

src/libraries/Native/Unix/Common/pal_atomic.h

src/libraries/Native/Unix/System.Globalization.Native/pal_calendarData.c

src/coreclr/src/vm/dllimport.cpp

safern · 2020-03-20T22:45:32Z

It seems like finally CI is almost green. Can I get another review on this please? 😄

jkotas · 2020-03-20T23:39:49Z

src/libraries/Native/Unix/System.Globalization.Native/pal_timeZoneInfo.c

@@ -4,7 +4,10 @@
 //

 #include <stdint.h>
+
+#if defined(TARGET_UNIX)
 #include <unistd.h>


Is this header actually needed on Unix?

Not sure. I’ll double check and if not I’ll remove it.

safern · 2020-03-21T07:32:26Z

It seems like the failures are: #32377 — I’ve also seen them fail in some recent PRs.

Dotnet-GitSync-Bot added the area-System.Globalization label Mar 18, 2020

safern added this to To do in ICU on Windows, Mobile, and Wasm via automation Mar 18, 2020

safern commented Mar 18, 2020

View reviewed changes

src/coreclr/src/dlls/mscoree/coreclr/CMakeLists.txt Show resolved Hide resolved

safern commented Mar 18, 2020

View reviewed changes

src/libraries/Native/AnyOS/System.Globalization.Native/CMakeLists.txt Outdated Show resolved Hide resolved

safern commented Mar 18, 2020

View reviewed changes

src/libraries/Native/AnyOS/System.Globalization.Native/configure.cmake Outdated Show resolved Hide resolved

safern commented Mar 18, 2020

View reviewed changes

src/libraries/Native/AnyOS/System.Globalization.Native/pal_icushim.c Outdated Show resolved Hide resolved

safern commented Mar 18, 2020

View reviewed changes

src/libraries/Native/AnyOS/System.Globalization.Native/pal_locale.h Outdated Show resolved Hide resolved

safern moved this from To do to In progress in ICU on Windows, Mobile, and Wasm Mar 18, 2020