Locale native test fails on Unix

[thezim]

Steps to reproduce

Run build per documentation.

Expected behavior

Builds successfully.

Actual behavior

Build fails with the following.

[----------] 1 test from LocaleTest
[ RUN      ] LocaleTest.Success
/Users/foobaruser/Source/PowerShell/src/libpsl-native/test/test-locale.cpp:20: Failure
Value of: "US-ASCII"
Expected: nl_langinfo(0)
Which is: "UTF-8"
[  FAILED  ] LocaleTest.Success (0 ms)
[----------] 1 test from LocaleTest (0 ms total)

Environment data

macOS 10.12

[andyleejordan]

@thezim this isn't a build failure, this is a unit test failure. Can you set your locale to UTF-8? .NET Core isn't expected to work with any other locales currently (that said, ASCII is probably going to work, but still going to fail the sanity test).

[thezim]

My text encoding was set to "UTF-8" in the Terminal app. While research how to change the locale I came across this article which was helpful. Unchecking the Terminal > Preferences > Advanced > Set locale environment variables on startup corrected the issue.

That being said should the build documentation be update with this tidbit as Set locale environment variables on startup is checked by default in macOS 10.12 (Sierra). I know 10.12 isn't officially supported yet but this will be an issue moving forward. I do not have access to a 10.11 machine so I cant tell you what the default setting there is.

With Set locale environment variables on startup checked.

thezim@foobarhost ~ $ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="C"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=
thezim@foobarhost ~ $ 

With Set locale environment variables on startup unchecked.

thezim@foobarhost ~ $ locale
LANG="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_ALL=
thezim@foobarhost ~ $ 

[mwallner]

this issue generally affects all terminals (#3063) that do not use utf-8 .. as @andschwa mentioned .NET core currently relies on UTF-8 I can't really complain that much .. still not a clean test ..

[vors]

I actually have problem earlier during the build on 10.12

PS /PowerShell> Start-PSBuild -Clean
Cleaning your working directory. You can also do it with 'git clean -fdX'
Removing powershell.version
VERBOSE: Using configuration 'Linux'
VERBOSE: Top project directory is /PowerShell/src/powershell-unix
VERBOSE: Using framework 'netcoreapp1.1'
The variable cannot be validated because the value osx.10.12-x64 is not a valid value for the Runtime variable.
At /PowerShell/build.psm1:528 char:9
+         $Runtime = dotnet --info | % {
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : MetadataError: (:) [], ValidationMetadataException
    + FullyQualifiedErrorId : ValidateSetFailure

Could not determine Runtime Identifier, please update dotnet
At /PowerShell/build.psm1:535 char:13
+             Throw "Could not determine Runtime Identifier, please upd ...
+             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Could not deter...e update dotnet:String) [], RuntimeException
    + FullyQualifiedErrorId : Could not determine Runtime Identifier, please update dotnet

[mwallner]

@vors what you are describing matches the issue text but is not related with the error @thezim is having (build failure vs test failure after successful build)

[vors]

Fair point

[vors]

There was same problem reported on gitter
https://gist.github.com/mwallner/457bde412489830319be10212bfb91e7#file-psl-native-test-log

Looks like an incorrect test for me.
The locale is a machine-wide setting that could be not UTF8.

[mwallner]

@vors please remove macOS from the title (replace with Nix*-like OSes??) and add 'Linux' as label :-)

[andyleejordan]

@vors if the locale is not UTF-8, we have no guarantees that CoreCLR will work correctly. Much code depends on being able to marshal UTF-8 char* strings to C# UTF-16 strings. Can we verify how CoreCLR handles the case where the locale is not UTF-8?

[SteveL-MSFT]

Haven't seen an issue with this anymore