[mono] Use unsigned char when computing UTF8 string hashes #83273
Conversation
The C standard does not specify whether `char` is signed or unsigned, it is implementation defined. Apparently Android aarch64 makes a different choice than other platforms (at least macOS arm64 and Windows x64 give different results). Mono uses `mono_metadata_str_hash` in the AOT compiler and AOT runtime to optimize class name lookup. As a result, classes whose names include UTF-8 continuation bytes (with the high bit = 1) will hash differently in the AOT compiler and on the device. Fixes dotnet#82187 Fixes dotnet#78638
AOT compiler: Emits a second "class_name_table_debug" symbol that has all the class names and hashes as strings. AOT runtime: warns if a class is not found in the name cache
|
We will need to backport this to net7, net6 and mono 2020-02 |
|
I want to add a regression test here, but we don't run Android-arm64 device tests with AOT, AFAICT. We run libraries tests with the minijit, but we don't run runtime tests at all. I'm going to check if linux arm64 repros this issue - that might be good enough for the purposes of adding a regression test |
|
see the discussion here #82833 (comment) |
|
verified that linux-arm64 has |
lambdageek
added
the
NO-MERGE
|
Temporarily reverted #59772 and turned on AOT compilation for the runtime tests on Android devices. |
@lewing I don't totally follow how that's related. In this case the issue is not the project or assembly names, it's the names of classes inside an assembly. They're kind of related problems, but in this case even if all the filesystem names are ascii, there are still problems |
|
/azp run runtime-extra-platforms-android |
|
No pipelines are associated with this pull request. |
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
lambdageek
force-pushed
the
fix-gh-82187
branch
from
ac76044
to
275aea2
Compare
|
/azp run runtime-extra-platforms |
|
Azure Pipelines successfully started running 1 pipeline(s). |
This reverts commit 275aea2.
lambdageek
removed
the
NO-MERGE
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
/azp run sync-runtime-to-mono |
|
No pipelines are associated with this pull request. |
|
mono/mono |
|
(reassigned to |
Backport of dotnet/runtime#83273 to mono/mono `main` The C standard does not specify whether `char` is signed or unsigned, it is implementation defined. Apparently Android aarch64 makes a different choice than other platforms (at least macOS arm64 and Windows x64 give different results). Mono uses `mono_metadata_str_hash` in the AOT compiler and AOT runtime to optimize class name lookup. As a result, classes whose names include UTF-8 continuation bytes (with the high bit = 1) will hash differently in the AOT compiler and on the device. Contributes to dotnet/runtime#82187 Contributes to dotnet/runtime#78638
Backport of dotnet/runtime#83273 to mono/mono `2020-02` The C standard does not specify whether `char` is signed or unsigned, it is implementation defined. Apparently Android aarch64 makes a different choice than other platforms (at least macOS arm64 and Windows x64 give different results). Mono uses `mono_metadata_str_hash` in the AOT compiler and AOT runtime to optimize class name lookup. As a result, classes whose names include UTF-8 continuation bytes (with the high bit = 1) will hash differently in the AOT compiler and on the device. Contributes to dotnet/runtime#82187 Contributes to dotnet/runtime#78638
ghost
locked as resolved and limited conversation to collaborators
The C standard does not specify whether
charis signed or unsigned,it is implementation defined.
Apparently Android aarch64 makes a different choice than other
platforms (at least macOS arm64 and Windows x64 give different
results).
Mono uses
mono_metadata_str_hashin the AOT compiler and AOT runtimeto optimize class name lookup. As a result, classes whose names
include UTF-8 continuation bytes (with the high bit = 1) will hash
differently in the AOT compiler and on the device.
Fixes #82187
Fixes #78638
Also add
DEBUG_AOT_NAME_TABLEifdef (undef'd by default) for debugging name hashing issuesAOT compiler: Emits a second "class_name_table_debug" symbol that has all the class
names and hashes as strings.
AOT runtime: warns if a class is not found in the name cache