Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System.IO.Net5Compat.Tests and System.IO.Tests suddenly exiting with error 137 #100558

Open
carlossanlop opened this issue Apr 2, 2024 · 3 comments
Open

System.IO.Net5Compat.Tests and System.IO.Tests suddenly exiting with error 137 #100558

carlossanlop opened this issue Apr 2, 2024 · 3 comments
Labels
arch-x64 area-System.IO Known Build Error Use this to report build issues in the .NET Helix tab untriaged New issue has not been triaged by the area owner

Comments

@carlossanlop
Copy link
Member

carlossanlop commented Apr 2, 2024 •
edited by build-analysis bot

The System.IO.Net5Compat.Tests and the System.IO.Tests test processes are intermittengly getting killed on Linux shortly after starting, and the exit code is 137.

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=627407
Build error leg or test failing: System.IO.Net5Compat.Tests

Error Message

{
  "ErrorPattern": ["Starting:    System\\.IO\\.(Net5Compat\\.)?Tests", "exit code 137"],
  "BuildRetry" : true,
  "ExcludeConsoleLog" : false
}

System.IO.Net5Compat.Tests example

===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.IO.Net5Compat.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.IO.Net5Compat.Tests (found 679 of 685 test cases)
  Starting:    System.IO.Net5Compat.Tests (parallel test collections = on, max threads = 2)
./RunTests.sh: line 162:    25 Killed                  "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.IO.Net5Compat.Tests.runtimeconfig.json --depsfile System.IO.Net5Compat.Tests.deps.json xunit.console.dll System.IO.Net5Compat.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Tue Apr 2 20:20:02 UTC 2024 ----- exit code 137 ----------------------------------------------------------

System.IO.Test example

===========================================================================================================
/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: System.IO.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.IO.Tests (found 679 of 685 test cases)
  Starting:    System.IO.Tests (parallel test collections = on, max threads = 2)
./RunTests.sh: line 162:    25 Killed                  "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.IO.Tests.runtimeconfig.json --depsfile System.IO.Tests.deps.json xunit.console.dll System.IO.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Tue Apr 2 20:20:10 UTC 2024 ----- exit code 137 ----------------------------------------------------------

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=627407
Error message validated: [Starting: System\.IO\.(Net5Compat\.)?Tests exit code 137]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 4/2/2024 11:08:28 PM UTC

Report

Build Definition Test Pull Request
699686 dotnet/runtime System.IO.Tests.WorkItemExecution
699404 dotnet/runtime System.IO.Tests.WorkItemExecution #103119
699424 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
699114 dotnet/runtime System.IO.Tests.WorkItemExecution #103095
698950 dotnet/runtime System.IO.Tests.WorkItemExecution #103104
698893 dotnet/runtime System.IO.Tests.WorkItemExecution
697087 dotnet/runtime System.IO.Tests.WorkItemExecution #103036
698735 dotnet/runtime System.IO.Tests.WorkItemExecution #103094
698777 dotnet/runtime System.IO.Net5Compat.Tests.WorkItemExecution
698660 dotnet/runtime System.IO.Tests.WorkItemExecution #102523
698624 dotnet/runtime System.IO.Tests.WorkItemExecution #103089
697725 dotnet/runtime System.IO.Tests.WorkItemExecution #103044
698161 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
692933 dotnet/runtime System.IO.Tests.WorkItemExecution #102834
697996 dotnet/runtime System.IO.Tests.WorkItemExecution #103066
2466886 dotnet-runtime System.IO.Tests.WorkItemExecution #39678
697448 dotnet/runtime System.IO.Tests.WorkItemExecution
697437 dotnet/runtime System.IO.Tests.WorkItemExecution #103016
697354 dotnet/runtime System.IO.Tests.WorkItemExecution
697095 dotnet/runtime System.IO.Tests.WorkItemExecution
696945 dotnet/runtime System.IO.Net5Compat.Tests.WorkItemExecution #102558
696786 dotnet/runtime System.IO.Tests.WorkItemExecution #100334
696700 dotnet/runtime System.IO.Tests.WorkItemExecution #100506
696326 dotnet/runtime System.IO.Tests.WorkItemExecution #103012
2465724 dotnet-runtime System.IO.Tests.WorkItemExecution #39678
695886 dotnet/runtime System.IO.Tests.WorkItemExecution #102992
695704 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
695191 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
694875 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
692901 dotnet/runtime System.IO.Tests.WorkItemExecution #102498
694256 dotnet/runtime System.IO.Tests.WorkItemExecution
694168 dotnet/runtime System.IO.Tests.WorkItemExecution #102862
693855 dotnet/runtime System.IO.Tests.WorkItemExecution #102902
693887 dotnet/runtime System.IO.Net5Compat.Tests.WorkItemExecution #102558
693693 dotnet/runtime System.IO.Tests.WorkItemExecution #102808
693619 dotnet/runtime System.IO.Tests.WorkItemExecution
693336 dotnet/runtime System.IO.Tests.WorkItemExecution #102903
693328 dotnet/runtime System.IO.Tests.WorkItemExecution #102364
693312 dotnet/runtime System.IO.Tests.WorkItemExecution #102901
693305 dotnet/runtime System.IO.Tests.WorkItemExecution #102853
693156 dotnet/runtime System.IO.Tests.WorkItemExecution #102636
692912 dotnet/runtime System.IO.Tests.WorkItemExecution #102862
692755 dotnet/runtime System.IO.Tests.WorkItemExecution #100356
692540 dotnet/runtime System.IO.Tests.WorkItemExecution #102875
692575 dotnet/runtime System.IO.Tests.WorkItemExecution #102876
692493 dotnet/runtime System.IO.Tests.WorkItemExecution #102754
692511 dotnet/runtime System.IO.Net5Compat.Tests.WorkItemExecution #102558
692132 dotnet/runtime System.IO.Tests.WorkItemExecution #102862
692041 dotnet/runtime System.IO.Tests.WorkItemExecution #102795
691860 dotnet/runtime System.IO.Tests.WorkItemExecution #102852
691818 dotnet/runtime System.IO.Tests.WorkItemExecution #102846
691718 dotnet/runtime System.IO.Tests.WorkItemExecution #102261
691382 dotnet/runtime System.IO.Tests.WorkItemExecution #102379
691295 dotnet/runtime System.IO.Tests.WorkItemExecution #102315
691231 dotnet/runtime System.IO.Tests.WorkItemExecution #102828
691039 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
690819 dotnet/runtime System.IO.Tests.WorkItemExecution #102805
690656 dotnet/runtime System.IO.Tests.WorkItemExecution #102786
690582 dotnet/runtime System.IO.Tests.WorkItemExecution #102096
690468 dotnet/runtime System.IO.Tests.WorkItemExecution #102791
689470 dotnet/runtime System.IO.Tests.WorkItemExecution #102754
689748 dotnet/runtime System.IO.Tests.WorkItemExecution #102769
689490 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
689256 dotnet/runtime System.IO.Tests.WorkItemExecution #101938
688841 dotnet/runtime System.IO.Tests.WorkItemExecution #102628
688826 dotnet/runtime System.IO.Tests.WorkItemExecution #102727
688824 dotnet/runtime System.IO.Tests.WorkItemExecution #102726
688566 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
688469 dotnet/runtime System.IO.Tests.WorkItemExecution #102718
688116 dotnet/runtime System.IO.Tests.WorkItemExecution #102700
687947 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
687914 dotnet/runtime System.IO.Tests.WorkItemExecution #102498
687812 dotnet/runtime System.IO.Tests.WorkItemExecution #102695
687677 dotnet/runtime System.IO.Tests.WorkItemExecution #99183
687512 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
687437 dotnet/runtime System.IO.Tests.WorkItemExecution #102685
687411 dotnet/runtime System.IO.Tests.WorkItemExecution #100951
687264 dotnet/runtime System.IO.Tests.WorkItemExecution #102180
687041 dotnet/runtime System.IO.Tests.WorkItemExecution
687027 dotnet/runtime System.IO.Tests.WorkItemExecution #99183
687009 dotnet/runtime System.IO.Tests.WorkItemExecution #100951
687005 dotnet/runtime System.IO.Tests.WorkItemExecution #102364
686965 dotnet/runtime System.IO.Tests.WorkItemExecution #102379
686943 dotnet/runtime System.IO.Tests.WorkItemExecution #102592
686921 dotnet/runtime System.IO.Tests.WorkItemExecution #102667
686906 dotnet/runtime System.IO.Tests.WorkItemExecution
686264 dotnet/runtime System.IO.Tests.WorkItemExecution #102640
686579 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
686376 dotnet/runtime System.IO.Tests.WorkItemExecution #102484
686269 dotnet/runtime System.IO.Tests.WorkItemExecution #102639
686134 dotnet/runtime System.IO.Tests.WorkItemExecution #102604
686115 dotnet/runtime System.IO.Tests.WorkItemExecution
686008 dotnet/runtime System.IO.Tests.WorkItemExecution #102636
685888 dotnet/runtime System.IO.Tests.WorkItemExecution #102624
685866 dotnet/runtime System.IO.Tests.WorkItemExecution #102404
685587 dotnet/runtime System.IO.Tests.WorkItemExecution #102509
685572 dotnet/runtime System.IO.Tests.WorkItemExecution #102604
685469 dotnet/runtime System.IO.Tests.WorkItemExecution #102605
685382 dotnet/runtime System.IO.Tests.WorkItemExecution #102558
685238 dotnet/runtime System.IO.Tests.WorkItemExecution #101701
Displaying 100 of 208 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
9 40 208
@carlossanlop carlossanlop added area-System.IO arch-x64 runtime-coreclr specific to the CoreCLR runtime os-linux-musl Linux distributions using musl library. Known Build Error Use this to report build issues in the .NET Helix tab labels Apr 2, 2024
@dotnet-policy-service Dotnet Policy Service
Copy link
Contributor

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

@dotnet-policy-service dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Apr 2, 2024
@carlossanlop carlossanlop changed the title System.IO.Net5Compat.Tests suddenly exiting with error 137 System.IO.Net5Compat.Tests and System.IO.Tests suddenly exiting with error 137 Apr 2, 2024
@carlossanlop carlossanlop added os-linux Linux OS (any supported distro) runtime-mono specific to the Mono runtime labels Apr 2, 2024
@ericstj ericstj removed os-linux Linux OS (any supported distro) os-linux-musl Linux distributions using musl library. runtime-mono specific to the Mono runtime runtime-coreclr specific to the CoreCLR runtime labels Apr 12, 2024
@ericstj
Copy link
Member

ericstj commented Apr 12, 2024

@dotnet/area-system-io there are a lot of hits on this and relatively recent. It seems to me to be happening across many configurations. I think it's worth having a look.

@adamsitnik
Copy link
Member

The System.IO.Net5Compat.Tests and the System.IO.Tests test processes are intermittengly getting killed on Linux shortly after starting, and the exit code is 137.

137 means out of memory. We have not made any changes to 6.0 in System.IO, so I expect that either there was some infra change (like less memory available) or a bug was introduced in the product itself. The bug would be specific to Linux.

@carlossanlop is it possible to perform some kind of binary search based on the merged PRs and when it started to fail?

@elinor-fung elinor-fung mentioned this issue Apr 29, 2024
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 area-System.IO Known Build Error Use this to report build issues in the .NET Helix tab untriaged New issue has not been triaged by the area owner
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@carlossanlop @adamsitnik @ericstj