Locale regression test failures

[gco]

The regression tests fail without any locale settings or with "C" or "en_us.UTF-8".

env LANG=en_us.UTF-8 LC_CTYPE=en_us.UTF-8 LC_MESSAGES=en_us.UTF-8 ./mvnw integration-test
...
[ERROR]   TextAnalyzerTest.defaultEncoding:61 expected:<[ASCII]> but was:<[UTF8]>
[ERROR]   TextAnalyzerTest.resetsStreamOnShortInput:71 expected:<[ASCII]> but was:<[UTF8]>
...

Versions:

SunOS 5.11 11.3 sun4v sparc sun4v

java version "1.8.0_141"
Java(TM) SE Runtime Environment (build 1.8.0_141-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode)

[vladak]

I can reproduce this on Solaris 11.3 by running:

export LANG=C
export LC_ALL=C

so the locale output looks like this:

LANG=C
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_ALL=C

With my default locale:

LANG=en_US.UTF-8
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=en_US.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE=en_US.UTF-8
LC_MONETARY=en_US.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_ALL=en_US.UTF-8

the tests pass so I guess this is a problem in the environment or in how the environment variables are set.

Side note: on Solaris 11.3 I had to edit mvnw to run bash instead of /bin/sh since the script relies on the local keyword and /bin/sh is symlinked to ksh93 which does not seem to understand the keyword. That's another reason to get rid of the Maven wrapper script.

[vladak]

Actually, looking at the defaultEncoding() test once again, the defaultEncoding is determined during startup:

43      private final String defaultEncoding = new InputStreamReader(new ByteArrayInputStream(new byte[0])).getEncoding();

and then in the TestableTextAnalyzer (which inherits TextAnalyzer):

129              try (Reader r = getReader(src.getStream())) {
130                  encoding = ((InputStreamReader) r).getEncoding();

where getReader() is defined in TextAnalyzer like this:

90      protected Reader getReader(InputStream stream) throws IOException {
91          // SRCROOT is read with UTF-8 as a default.
92          return IOUtils.createBOMStrippedReader(stream,
93              StandardCharsets.UTF_8.name());
94      }

so running this test with the C locale will always make it fail as getReader() hard-codes the encoding to UTF-8 after cset e8ef5ee (Fix to consistently default to UTF-8 for SRCROOT reader).

So rather than trying to figure out how to pass UTF-8 locale to Maven, the test should be fixed.

[vladak]

And by fixing, I mean ripping out the encoding checks since they no longer match the reality.