Timeout in System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers

[lewing]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=646861
Build error leg or test failing: System.Text.Json.Tests.WorkItemExecution
Pull request: #101140

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": [ "Takes very long to run on interpreter.", "System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers", "['System.Text.Json.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]"],
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=646861
Error message validated: [Takes very long to run on interpreter. System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers ['System.Text.Json.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/17/2024 5:06:43 PM UTC

Report

Build	Definition	Test	Pull Request
678721	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
678685	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#102362
678603	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#102357
678577	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
678427	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#102351
671701	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
664107	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
664022	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101828
663982	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
663692	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101818
663649	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
663382	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101807
661263	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
658219	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
658149	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101645
656450	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101573
655218	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101318
655926	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101540
656399	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101569
656378	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#99829
656370	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
656345	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101568
655031	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101501
656316	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101513
656292	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101566
655011	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101221
656230	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101188
655417	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101435
656221	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101450
656213	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101381
656209	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100266
655408	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101469
656161	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101531
656108	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101555
656100	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101451
656090	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101294
656061	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
656049	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100141
656029	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101020
656005	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100823
655951	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101545
655948	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101545
655934	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101122
655920	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101252
655885	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101378
655832	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101330
655807	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101492
655649	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101529
655753	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101133
655731	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101053
655692	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101531
655689	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101478
655669	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101392
655645	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101250
655619	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101531
655583	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
655571	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101392
655420	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101480
655504	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101460
655451	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
655445	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101525
655401	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101378
655365	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101450
655360	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101451
655274	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101122
655262	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101460
654452	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100743
654860	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100094
653807	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101442
654571	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101291
654787	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100949
654744	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101330
653322	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101188
654715	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101492
654688	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100918
654316	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101469
654664	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101449
654653	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100697
654649	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101250
654644	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101447
654588	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101480
654556	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101478
654532	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100775
654504	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution
654434	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101474
654382	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101192
652161	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101294
654360	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101461
654330	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101101
654212	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101460
654303	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101468
654293	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101451
653784	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100094
654135	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101457
653837	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101445
653747	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#95830
653721	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#100993
654018	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101451
653949	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101416
653946	dotnet/runtime	System.Text.Json.Tests.WorkItemExecution	#101450
Displaying 100 of 200 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	5	200

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[lewing]

See the discussion in #59678 for context

[lewing]

This was written to try to catch
https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-101122-merge-004d05ddb3b041639a/System.Text.Json.Tests/1/console.8d81d47f.log?helixlogtype=result

/datadisks/disk1/work/9B2B0894/w/B43409F0/e /datadisks/disk1/work/9B2B0894/w/B43409F0/e
  Discovering: System.Text.Json.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Text.Json.Tests (found 7405 of 7466 test cases)
  Starting:    System.Text.Json.Tests (parallel test collections = on [2 threads], stop on fail = off)
    System.Text.Json.Tests.Utf8JsonReaderTests.TestDepth(depth: 256) [SKIP]
      Takes very long to run on interpreter.
    System.Text.Json.Tests.Utf8JsonReaderTests.TestDepth(depth: 512) [SKIP]
      Takes very long to run on interpreter.
   System.Text.Json.Tests: [Long Running Test] 'System.Text.Json.Tests.Utf8JsonWriterTests.Writing3MBBase64Bytes', Elapsed: 00:05:42
   System.Text.Json.Tests: [Long Running Test] 'System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers', Elapsed: 00:02:01
   System.Text.Json.Tests: [Long Running Test] 'System.Text.Json.Tests.Utf8JsonWriterTests.WriteNumbers', Elapsed: 00:04:01
Killed
['System.Text.Json.Tests' END OF WORK ITEM LOG: Command timed out, and was killed]

as explicitly as possible but there are a lot of other forms of timeouts for wasm

[lewing]

@BrzVlad the particular failure this issue seems to catch looks like it is specfic to the desktop interpreter, can you take a look?

[dotnet-policy-service]

Tagging subscribers to this area: @BrzVlad, @kotlarmilos
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis
See info in area-owners.md if you want to be subscribed.

[BrzVlad]

I did a few runs locally and I'm not seeing anything suspicious. It just looks like a heavy suite that takes a long time to run. Also I'm seeing that all these failures are on debug builds. While for jit/aot, debug build doesn't have a big impact, the interpreter execution engine is written in C and it can easily get 3x slower in debug build.

I'm not sure if it easy to do, but I think this test suite should be disabled on interpreter debug builds

[BrzVlad]

Disabled suite in debug runtime builds in /github.com//pull/101483

[lewing]

reopening since a few builds weren't against the tip of main and hit this, feel free to close again next week