Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threadpool test CooperativeBlockingCanCreateThreadsFaster failing on Mac #66852

Closed
runfoapp bot opened this issue Mar 18, 2022 · 5 comments · Fixed by #67399
Closed

Threadpool test CooperativeBlockingCanCreateThreadsFaster failing on Mac #66852

runfoapp bot opened this issue Mar 18, 2022 · 5 comments · Fixed by #67399
Labels
area-System.Threading disabled-test The test is disabled in source code against the issue

Comments

@runfoapp
Copy link

runfoapp bot commented Mar 18, 2022

Runfo Tracking Issue: Threadpool test CooperativeBlockingCanCreateThreadsFaster failing on Mac

Build Definition Kind Run Name Console Core Dump Test Results Run Client
1675090 runtime PR 66688 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
0 0 0
@agocke agocke added area-System.Threading blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' labels Mar 18, 2022
@ghost
Copy link

ghost commented Mar 18, 2022

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

Runfo Tracking Issue: Threadpool test CooperativeBlockingCanCreateThreadsFaster failing on Mac

Build Definition Kind Run Name Console Core Dump Test Results Run Client
1670193 runtime PR 62843 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1670162 runtime PR 66819 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1670130 runtime Rolling net7.0-OSX-Release-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1670090 runtime PR 66814 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1669532 runtime PR 65948 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1669442 runtime PR 57610 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1669442 runtime PR 57610 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1669249 runtime PR 61776 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1669093 runtime PR 66742 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1669052 runtime Rolling net7.0-OSX-Release-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1669052 runtime Rolling net7.0-OSX-Release-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1669038 runtime PR 66147 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1669038 runtime PR 66147 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1669006 runtime PR 66784 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1668967 runtime PR 66783 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1668791 runtime PR 66777 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1668730 runtime PR 66748 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1668730 runtime PR 66748 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1668686 runtime Rolling net7.0-OSX-Release-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1668647 runtime PR 65101 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1668647 runtime PR 65101 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1668553 runtime PR 66765 net7.0-OSX-Debug-x64-Mono_release-OSX.1200.Amd64.Open console.log runclient.py
1668553 runtime PR 66765 net7.0-Linux-Debug-arm64-Mono_release-(Ubuntu.1804.ArmArch.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-16.04-helix-arm64v8-20210106155927-56c6673 console.log core dump runclient.py
1668553 runtime PR 66765 net7.0-Linux-Debug-x64-Mono_release-(Centos.7.Amd64.Open)Ubuntu.1604.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-mlnet-helix-20210714125435-dde38af console.log core dump runclient.py
1668553 runtime PR 66765 net7.0-Linux-Debug-x64-Mono_release-RedHat.7.Amd64.Open console.log core dump runclient.py
1668553 runtime PR 66765 net7.0-Linux-Debug-x64-Mono_release-(Debian.10.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-20210304164434-56c6673 console.log core dump runclient.py
1668553 runtime PR 66765 net7.0-Linux-Debug-x64-Mono_release-Ubuntu.1804.Amd64.Open console.log core dump runclient.py
1668553 runtime PR 66765 net7.0-Linux-Debug-x64-mono_interpreter_release-(Debian.10.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-10-helix-amd64-20210304164434-56c6673 console.log core dump runclient.py
1668347 runtime PR 66749 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1668063 runtime Rolling net7.0-OSX-Release-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1667940 runtime PR 66754 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1667911 runtime PR 66677 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1667860 runtime PR 66753 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1667860 runtime PR 66753 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1667853 runtime PR 66752 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1667849 runtime PR 66685 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1667835 runtime PR 66486 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1667479 runtime PR 57610 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1666625 runtime PR 66711 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1666600 runtime PR 66650 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1666532 runtime PR 66712 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1666474 runtime PR 66710 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1666422 runtime PR 66605 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1666398 runtime PR 66702 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1666323 runtime PR 66703 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1666300 runtime PR 63894 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1666212 runtime PR 66698 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1666153 runtime PR 62843 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665804 runtime PR 66676 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665804 runtime PR 66676 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665787 runtime PR 65249 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665769 runtime PR 66693 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665698 runtime PR 66688 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665677 runtime PR 66455 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665575 runtime PR 66677 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665552 runtime PR 57610 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665321 runtime PR 66681 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665312 runtime PR 66131 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665277 runtime PR 66619 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665097 runtime PR 66666 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1665052 runtime PR 65994 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1665007 runtime PR 66434 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1664839 runtime PR 60507 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1664310 runtime PR 66609 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py
1663789 runtime PR 66528 net7.0-OSX-Debug-x64-CoreCLR_release-OSX.1200.Amd64.Open console.log runclient.py
1663784 runtime PR 66479 net7.0-OSX-Debug-x64-CoreCLR_checked-OSX.1200.Amd64.Open console.log runclient.py

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
12 53 53
Author: runfoapp[bot]
Assignees: -
Labels:

area-System.Threading, blocking-clean-ci

Milestone: -

@agocke
Copy link
Member

agocke commented Mar 18, 2022

@mangod9 This test has failed the last three rolling builds, so I think we should consider disabling in MacOS if the failure cause isn't obvious

@danmoseley
Copy link
Member

@kouvel #66755 looks like a dupe. but in there you mention mono, it's mostly coreclr.

@kouvel
Copy link
Member

kouvel commented Mar 19, 2022

I'm trying a fix in #66765. I'll close #66755 as a dupe.

@agocke
Copy link
Member

agocke commented Mar 22, 2022

Test was disabled by #66988, so this is no longer blocking

@agocke agocke removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Mar 22, 2022
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Mar 22, 2022
@agocke agocke added the disabled-test The test is disabled in source code against the issue label Mar 23, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Mar 31, 2022
kouvel added a commit to kouvel/runtime that referenced this issue Mar 31, 2022
- The test verifies that cooperative blocking in `Task.Wait()` causes threads to be injected quickly enough that it does not time out due to starvation
- Cooperative blocking checks memory usage and limit, and beyond a threshold of memory usage does not inject threads quickly
- Following a build of the runtime repo, there are several `dotnet` processes that remain running, each using several 100s of MBs and one using ~1.3 GB of memory
- If the test runs soon after the build, before those processes exit, it's possible for the reported memory usage to be high enough that the test would fail
- Added a config var to ignore memory usage and used it in the test

Fixes dotnet#66852
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Mar 31, 2022
kouvel added a commit that referenced this issue Apr 26, 2022
- The test verifies that cooperative blocking in `Task.Wait()` causes threads to be injected quickly enough that it does not time out due to starvation
- Cooperative blocking checks memory usage and limit, and beyond a threshold of memory usage does not inject threads quickly
- Following a build of the runtime repo, there are several `dotnet` processes that remain running, each using several 100s of MBs and one using ~1.3 GB of memory
- If the test runs soon after the build, before those processes exit, it's possible for the reported memory usage to be high enough that the test would fail
- Added a config var to ignore memory usage and used it in the test

Fixes #66852
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Apr 26, 2022
@dotnet dotnet locked as resolved and limited conversation to collaborators May 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Threading disabled-test The test is disabled in source code against the issue
Projects
None yet
3 participants