Fix utf8_to_utf16 and utf16_to_utf8 for s390x

[saitama951]

The converters return incorrect string when MINIPAL_TREAT_AS_LITTLE_ENDIAN is set true on s390x

see: #128394 (comment)

[saitama951]

@Dotnet-s390x build

[saitama951]

cc: @jkotas

[Dotnet-s390x]

Build Queued..

To cancel the current build, please comment:

@Dotnet-s390x cancel

[am11]

@saitama951, was there any PAL test failing before this change? If not, could we add a case or two there which exercises it (that would fail without this fix)? Possible place of addition:
src/coreclr/pal/tests/palsuite/locale_info/{MultiByteToWideChar,WideCharToMultiByte}/test{1,2,3,4,5}/test{1,2,3,4,5}.cpp

It would help gain some confidence in the future, if someone refactors this code (e.g. with SIMD etc.). We refactored it in 2023 and seeing that we are only finding it out now is a bit worrying. :/

[Dotnet-s390x]

Build Successful
Please check the build logs: http://148.100.85.217:8080/job/dotnet-builds/80/console.

[saitama951]

@am11 I think the intent of MultiByteToWideChar, WideCharToMultiByte are supposed to be in native endian which worked fine even before this patch, It is only when minipal is forced to convert strings in LE order in big endian system that this failed. I think the eventpipe stuff forces things to write down into a .nettrace in little-endian order.

[am11]

You can cherry pick this commit main...am11:runtime:chore/utf-bigendian-tests. I don't have access to bigendian machine right now, but if it fails on main branch and passes with PR, it would suffice for minipal API coverage.

[jkotas]

You can cherry pick this commit main...am11:runtime:chore/utf-bigendian-tests

I doubt that the Win32 PAL tests work on s390x. We can look into the testing separately.

[am11]

You can cherry pick this commit main...am11:runtime:chore/utf-bigendian-tests

I doubt that the Win32 PAL tests work on s390x. We can look into the testing separately.

They work just fine #128394 (comment).

[saitama951]

@am11
Running tests...

......................................................................................Testing for 64 Bit Platforms 
............................................................................................................minipal_convert_utf8_to_utf16 with MINIPAL_TREAT_AS_LITTLE_ENDIAN mismatch at 0: got 0x0041 expected 0x4100


FAILED: locale_info/MultiByteToWideChar/test4/paltest_multibytetowidechar_test4. Exit code: 1

....minipal_get_length_utf16_to_utf8 with MINIPAL_TREAT_AS_LITTLE_ENDIAN returned 97, expected 33


FAILED: locale_info/WideCharToMultiByte/test5/paltest_widechartomultibyte_test5. Exit code: 1

..........................Global Counter Value at the end of the test 0 
..........................

after applying this patch I started to get this failure

..minipal_get_length_utf16_to_utf8 with MINIPAL_TREAT_AS_LITTLE_ENDIAN returned 97, expected 33

I have made endian changes to GetByteCount as well and with that I get

Finished running PAL tests.

PAL Test Results:
  Passed: 305
  Failed: 0

[saitama951]

@Dotnet-s390x build

[Dotnet-s390x]

Build Queued..

To cancel the current build, please comment:

@Dotnet-s390x cancel

[Dotnet-s390x]

Please check the build logs: http://148.100.85.217:8080/job/dotnet-builds/81/console.

[am11]

LGTM, thank you! 👍