Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar #89094

Closed
Jozkee opened this issue Jul 18, 2023 · 6 comments · Fixed by #89152
Closed

Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar #89094

Jozkee opened this issue Jul 18, 2023 · 6 comments · Fixed by #89152
Labels
area-System.Text.Json blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab

Comments

@Jozkee
Copy link
Member

Jozkee commented Jul 18, 2023 •
edited by build-analysis bot

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=344043
Build error leg or test failing: System.Text.Json.Tests.WorkItemExecution
Pull request: #89036

Error Blob

{
  "ErrorMessage": "System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Reproduction Steps

While adding support for Half to System.Text.Json (#88962), I noticed one test triggers the assertion failure consistently on macOS-12.4.
Note: This didn't repro for me locally on macOS 13.4.1 arm64.

The test attempts to deserialize floting-point literals (NaN, infinity, -Infinity) in multiple casings/variants that are invalid for S.T.Json. Error happens on line 883.

runtime/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs

Lines 848 to 883 in 43fe8a8

[Theory]
[InlineData("naN")]
[InlineData("Nan")]
[InlineData("NAN")]
[InlineData("+Infinity")]
[InlineData("+infinity")]
[InlineData("infinity")]
[InlineData("infinitY")]
[InlineData("INFINITY")]
[InlineData("+INFINITY")]
[InlineData("-infinity")]
[InlineData("-infinitY")]
[InlineData("-INFINITY")]
[InlineData(" NaN")]
[InlineData("NaN ")]
[InlineData(" Infinity")]
[InlineData(" -Infinity")]
[InlineData("Infinity ")]
[InlineData("-Infinity ")]
[InlineData("a-Infinity")]
[InlineData("NaNa")]
[InlineData("Infinitya")]
[InlineData("-Infinitya")]
#pragma warning disable xUnit1025 // Theory method 'FloatingPointConstants_Fail' on test class 'NumberHandlingTests' has InlineData duplicate(s)
[InlineData("\u006EaN")] // "naN"
[InlineData("\u0020Inf\u0069ni\u0074y")] // " Infinity"
[InlineData("\u002BInf\u0069nity")] // "+Infinity"
#pragma warning restore xUnit1025
public async Task FloatingPointConstants_Fail(string testString)
{
string testStringAsJson = $@"""{testString}""";
string testJson;
#if NETCOREAPP
testJson = @$"{{""HalfNumber"":{testStringAsJson}}}";
await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionsAllowFloatConstants));
await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionReadFromStr));

Helix log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-88962-merge-9757e201f6374e20b5/System.Text.Json.Tests/1/console.78feb75a.log?helixlogtype=result

Call stack:

/private/tmp/helix/working/9EC208C7/w/A9F108F7/e /private/tmp/helix/working/9EC208C7/w/A9F108F7/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 7160 of 7231 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 6)
Process terminated. Assertion failed.
   at System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar(Byte& charA, Int32 lengthA, Byte& charB, Int32 lengthB) in /_/src/libraries/System.Private.CoreLib/src/System/Globalization/Ordinal.Utf8.cs:line 308
   at System.Number.TryParseFloat[TChar,TFloat](ReadOnlySpan`1 value, NumberStyles styles, NumberFormatInfo info, TFloat& result) in /_/src/libraries/System.Private.CoreLib/src/System/Number.Parsing.cs:line 1229
   at System.Half.TryParse(ReadOnlySpan`1 utf8Text, NumberStyles style, IFormatProvider provider, Half& result) in /_/src/libraries/System.Private.CoreLib/src/System/Half.cs:line 2238
   at System.Text.Json.Serialization.Converters.HalfConverter.TryParse(ReadOnlySpan`1 buffer, Half& result) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 197
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadCore(Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 50
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadNumberWithCustomHandling(Utf8JsonReader& reader, JsonNumberHandling handling, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 115
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 193
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonPropertyInfoOfT.cs:line 308
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectDefaultConverter.cs:line 49
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 258
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs:line 51
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.ReadHelper.cs:line 22
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Span.cs:line 160
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 443
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 55
   at System.Text.Json.Serialization.Tests.JsonSerializerWrapper.StringSerializerWrapper.DeserializeWrapper[T](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/JsonSerializerWrapper.Reflection.cs:line 129
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<<FloatingPointConstants_Fail>b__1>d.MoveNext() in /_/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs:line 883
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) in /_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:line 38
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<FloatingPointConstants_Fail>b__1()

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=344043
Error message validated: System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/18/2023 11:30:55 PM UTC

Report

Build Definition Test Pull Request
344444 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89153
344421 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89154
344427 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89155
344398 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89151
344340 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #87438
344330 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89061
344321 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89147
344317 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89123
344290 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #87847
344296 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89102
344284 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89144
344268 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89130
344261 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #85328
344239 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #86089
344207 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89129
344182 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89135
344136 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #87656
344099 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89064
344092 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89116
344056 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89123
344043 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #89036
344049 dotnet/runtime System.Text.Json.Tests.WorkItemExecution
343994 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #87438
343982 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #85328
343063 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #88962
342911 dotnet/runtime System.Text.Json.Tests.WorkItemExecution #88962

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
25 26 26
@Jozkee Jozkee added blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab labels Jul 18, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jul 18, 2023
@ghost
Copy link

ghost commented Jul 18, 2023

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

Issue Details

Error Blob

{
  "ErrorMessage": "System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": true
}

Reproduction Steps

While adding support for Half to System.Text.Json (#88962), I noticed one test triggers the assertion failure consistently on OSX.

The test attempts to deserialize floting-point literals (NaN, infinity, -Infinity) in multiple casings/variants that are invalid for S.T.Json. Error happens on line 883.

runtime/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs

Lines 848 to 883 in 43fe8a8

[Theory]
[InlineData("naN")]
[InlineData("Nan")]
[InlineData("NAN")]
[InlineData("+Infinity")]
[InlineData("+infinity")]
[InlineData("infinity")]
[InlineData("infinitY")]
[InlineData("INFINITY")]
[InlineData("+INFINITY")]
[InlineData("-infinity")]
[InlineData("-infinitY")]
[InlineData("-INFINITY")]
[InlineData(" NaN")]
[InlineData("NaN ")]
[InlineData(" Infinity")]
[InlineData(" -Infinity")]
[InlineData("Infinity ")]
[InlineData("-Infinity ")]
[InlineData("a-Infinity")]
[InlineData("NaNa")]
[InlineData("Infinitya")]
[InlineData("-Infinitya")]
#pragma warning disable xUnit1025 // Theory method 'FloatingPointConstants_Fail' on test class 'NumberHandlingTests' has InlineData duplicate(s)
[InlineData("\u006EaN")] // "naN"
[InlineData("\u0020Inf\u0069ni\u0074y")] // " Infinity"
[InlineData("\u002BInf\u0069nity")] // "+Infinity"
#pragma warning restore xUnit1025
public async Task FloatingPointConstants_Fail(string testString)
{
string testStringAsJson = $@"""{testString}""";
string testJson;
#if NETCOREAPP
testJson = @$"{{""HalfNumber"":{testStringAsJson}}}";
await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionsAllowFloatConstants));
await Assert.ThrowsAsync<JsonException>(async () => await Serializer.DeserializeWrapper<StructWithNumbers>(testJson, s_optionReadFromStr));

Helix log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-88962-merge-9757e201f6374e20b5/System.Text.Json.Tests/1/console.78feb75a.log?helixlogtype=result

Call stack:

/private/tmp/helix/working/9EC208C7/w/A9F108F7/e /private/tmp/helix/working/9EC208C7/w/A9F108F7/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 7160 of 7231 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on, max threads = 6)
Process terminated. Assertion failed.
   at System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar(Byte& charA, Int32 lengthA, Byte& charB, Int32 lengthB) in /_/src/libraries/System.Private.CoreLib/src/System/Globalization/Ordinal.Utf8.cs:line 308
   at System.Number.TryParseFloat[TChar,TFloat](ReadOnlySpan`1 value, NumberStyles styles, NumberFormatInfo info, TFloat& result) in /_/src/libraries/System.Private.CoreLib/src/System/Number.Parsing.cs:line 1229
   at System.Half.TryParse(ReadOnlySpan`1 utf8Text, NumberStyles style, IFormatProvider provider, Half& result) in /_/src/libraries/System.Private.CoreLib/src/System/Half.cs:line 2238
   at System.Text.Json.Serialization.Converters.HalfConverter.TryParse(ReadOnlySpan`1 buffer, Half& result) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 197
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadCore(Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 50
   at System.Text.Json.Serialization.Converters.HalfConverter.ReadNumberWithCustomHandling(Utf8JsonReader& reader, JsonNumberHandling handling, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Value/HalfConverter.cs:line 115
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 193
   at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonPropertyInfoOfT.cs:line 308
   at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Converters/Object/ObjectDefaultConverter.cs:line 49
   at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.cs:line 258
   at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, JsonSerializerOptions options, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonConverterOfT.ReadCore.cs:line 51
   at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/Metadata/JsonTypeInfoOfT.ReadHelper.cs:line 22
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 utf8Json, JsonTypeInfo`1 jsonTypeInfo, Nullable`1 actualByteCount) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.Span.cs:line 160
   at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 443
   at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/src/System/Text/Json/Serialization/JsonSerializer.Read.String.cs:line 55
   at System.Text.Json.Serialization.Tests.JsonSerializerWrapper.StringSerializerWrapper.DeserializeWrapper[T](String json, JsonSerializerOptions options) in /_/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/JsonSerializerWrapper.Reflection.cs:line 129
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<<FloatingPointConstants_Fail>b__1>d.MoveNext() in /_/src/libraries/System.Text.Json/tests/Common/NumberHandlingTests.cs:line 883
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine) in /_/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:line 38
   at System.Text.Json.Serialization.Tests.NumberHandlingTests.<>c__DisplayClass39_0.<FloatingPointConstants_Fail>b__1()
Author: Jozkee
Assignees: -
Labels:

area-System.Text.Json, blocking-clean-ci, Known Build Error
Milestone: -

@Jozkee
Copy link
Member Author

Jozkee commented Jul 18, 2023

cc @tannergooding

@Jozkee Jozkee changed the title Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar on OSX only Jul 18, 2023
@elinor-fung
Copy link
Member

Recent PRs are consistently hitting this on Linux and Windows as well: linux, windows

@elinor-fung elinor-fung changed the title Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar on OSX only Half.TryParse assertion failure on System.Globalization.Ordinal.EqualsIgnoreCaseUtf8_Scalar Jul 18, 2023
@EgorBo
Copy link
Member

EgorBo commented Jul 18, 2023 •
edited

Sounds like it's from #86875 (at least it touched Ordinal.Utf8.cs and various parsing logic including Half)

@markples
Copy link
Member

adding some keywords to help github issue search (?)

EqualsIgnoreCaseUtf8_Scalar
Json.Tests
System.Text.Json.SourceGeneration.Roslyn4.4.Tests
Roslyn4.4.Tests
Roslyn4

@EgorBo EgorBo mentioned this issue Jul 19, 2023
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 19, 2023
@EgorBo
Copy link
Member

EgorBo commented Jul 19, 2023 •
edited

Looks like #88962 is the PR that broke CI by uncovering an existing bug, it seems that the assert was noticed and ignored on macOS 8c3e143

@ghost ghost removed in-pr There is an active PR which will close this issue when it is merged untriaged New issue has not been triaged by the area owner labels Jul 19, 2023
@tarekgh tarekgh mentioned this issue Jul 19, 2023
Closed
@hoyosjs hoyosjs mentioned this issue Jul 19, 2023
Merged
@dotnet dotnet locked as resolved and limited conversation to collaborators Aug 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-System.Text.Json blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' Known Build Error Use this to report build issues in the .NET Helix tab
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix EqualsIgnoreCaseUtf8_Scalar EgorBo/runtime-1
4 participants
@EgorBo @Jozkee @markples @elinor-fung