SystemEvents tests are failing (and hanging)

[ericstj]

From @BruceForstall in https://github.com/dotnet/coreclr/issues/17582

Many arm32 Windows corefx jobs in the CI are hanging, apparently on Microsoft.Win32.SystemEvents.Tests, e.g.:

https://ci.dot.net/job/dotnet_coreclr/job/master/view/arm/job/jitstress/job/arm_cross_checked_windows_nt_corefx_baseline_tst/11/console
https://ci.dot.net/job/dotnet_coreclr/job/master/view/arm/job/jitstress/job/arm_cross_checked_windows_nt_corefx_jitstress1_tst/11/console
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/arm_cross_checked_windows_nt_corefx_jitstressregs2_tst/9/console
https://ci.dot.net/job/dotnet_coreclr/job/master/view/arm/job/jitstress/job/arm_cross_checked_windows_nt_corefx_jitstressregs0x1000_tst/10/console

They also show failures before the hang.

Sample output:

22:52:06 Running: C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\tests\Microsoft.Win32.SystemEvents.Tests\netcoreapp-Windows_NT-Release-arm\RunTests.cmd C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm
22:52:06 Using C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm as the test runtime folder.
22:52:06 Executing in C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\tests\Microsoft.Win32.SystemEvents.Tests\netcoreapp-Windows_NT-Release-arm\ 
22:52:06 ----- start 22:52:08.17 ===============  To repro directly: ===================================================== 
22:52:06 pushd C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\tests\Microsoft.Win32.SystemEvents.Tests\netcoreapp-Windows_NT-Release-arm\
22:52:06 call C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm\dotnet.exe xunit.console.netcore.exe Microsoft.Win32.SystemEvents.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing
22:52:06 popd
22:52:06 ===========================================================================================================
22:52:06 
22:52:06 C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\tests\Microsoft.Win32.SystemEvents.Tests\netcoreapp-Windows_NT-Release-arm>call C:\Users\robox\j\workspace\arm_cross_che---eb7b805d\_\fx\bin\testhost\netcoreapp-Windows_NT-Release-arm\dotnet.exe xunit.console.netcore.exe Microsoft.Win32.SystemEvents.Tests.dll  -xml testResults.xml -notrait Benchmark=true -notrait category=nonnetcoreapptests -notrait category=nonwindowstests  -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing 
22:52:07 xUnit.net console test runner (32-bit .NET Core)
22:52:07 Copyright (C) 2014 Outercurve Foundation.
22:52:07 
22:52:07 Discovering: Microsoft.Win32.SystemEvents.Tests
22:52:09 Discovered:  Microsoft.Win32.SystemEvents.Tests
22:52:10 Starting:    Microsoft.Win32.SystemEvents.Tests
22:52:23 
   Microsoft.Win32.SystemEventsTests.LowMemoryTests.SignalsEventsAsynchronouslyOnMessage [FAIL]
22:52:23       Assert.True() Failure
22:52:23       Expected: True
22:52:23       Actual:   False
22:52:23       Stack Trace:
22:52:24             at Microsoft.Win32.SystemEventsTests.GenericEventTests.SignalsEventsAsynchronouslyOnMessage()
22:52:33    Microsoft.Win32.SystemEventsTests.CreateTimerTests.TimerElapsedSignaled [FAIL]
22:52:33       Assert.True() Failure
22:52:33       Expected: True
22:52:33       Actual:   False
22:52:33       Stack Trace:
22:52:33          D:\j\workspace\arm_cross_che---99792267\_\fx\src\Microsoft.Win32.SystemEvents\tests\SystemEvents.CreateTimer.cs(58,0): at Microsoft.Win32.SystemEventsTests.CreateTimerTests.TimerElapsedSignaled()
22:52:43    Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers [FAIL]
22:52:43       Assert.True() Failure
22:52:43       Expected: True
22:52:43       Actual:   False
22:52:43       Stack Trace:
22:52:43          D:\j\workspace\arm_cross_che---99792267\_\fx\src\Microsoft.Win32.SystemEvents\tests\SystemEvents.CreateTimer.cs(113,0): at Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers()
22:52:53    Microsoft.Win32.SystemEventsTests.DisplaySettingsTests.SignalsDisplayEventsAsynchronouslyOnDISPLAYCHANGE [FAIL]
22:52:53       Assert.True() Failure
22:52:53       Expected: True
22:52:53       Actual:   False
22:52:53       Stack Trace:
22:52:53          D:\j\workspace\arm_cross_che---99792267\_\fx\src\Microsoft.Win32.SystemEvents\tests\SystemEvents.DisplaySettings.cs(37,0): at Microsoft.Win32.SystemEventsTests.DisplaySettingsTests.SignalsDisplayEventsAsynchronouslyOnDISPLAYCHANGE()
22:53:03    Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsOnSameThreadAsOtherEvents [FAIL]
22:53:03       Assert.True() Failure
22:53:03       Expected: True
22:53:03       Actual:   False
22:53:03       Stack Trace:
22:53:03             at Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsOnSameThreadAsOtherEvents()
22:53:13    Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously [FAIL]
22:53:13       Assert.True() Failure
22:53:13       Expected: True
22:53:13       Actual:   False
22:53:13       Stack Trace:
22:53:13          D:\j\workspace\arm_cross_che---99792267\_\fx\src\Microsoft.Win32.SystemEvents\tests\SystemEvents.InvokeOnEventsThread.cs(22,0): at Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously()
06:44:16 
Build timed out (after 480 minutes). Marking the build as aborted.
06:44:16 

[ViktorHofer]

Just hit this on Windows.81.Amd64.Open-x64:Debug. I'm going to disable Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers.

C:\dotnetbuild\work\688d3729-d9a3-4600-99ca-8a7ecb1f3dae\Work\b7fe115e-6de6-4093-a46d-53c70eecebca\Exec>"C:\dotnetbuild\work\688d3729-d9a3-4600-99ca-8a7ecb1f3dae\Payload\dotnet.exe" xunit.console.dll Microsoft.Win32.SystemEvents.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=nonnetcoreapptests -notrait category=nonwindowstests -notrait category=IgnoreForCI -notrait category=failing -notrait category=OuterLoop  
  Discovering: Microsoft.Win32.SystemEvents.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Win32.SystemEvents.Tests (found 26 of 27 test cases)
  Starting:    Microsoft.Win32.SystemEvents.Tests (parallel test collections = on, max threads = 2)
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:02:11
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:04:11
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:06:11
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:08:11
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:10:11
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:12:12
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.CreateTimerTests.ConcurrentTimers', Elapsed: 00:14:12

https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F37414~2Fmerge/test~2Ffunctional~2Fcli~2F/20190503.23/workItem/Microsoft.Win32.SystemEvents.Tests/wilogs

[ViktorHofer]

@maryamariyan @ericstj are we concerned about these hangs and want to fix them for 3.0? Currently the milestone is set to Future.

[BruceForstall]

fyi, https://github.com/dotnet/coreclr/issues/22442 tracks timing out Win32.SystemEvents tests, and dotnet/coreclr#24201 disabled some of these.

[stephentoub]

Another failure here, on netfx:
https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F38506~2Fmerge/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190613.12/workItem/Microsoft.Win32.SystemEvents.Tests/wilogs

C:\dotnetbuild\work\09d4c5c7-d4ad-418a-bad6-86ef34d25081\Work\20c28d9f-1c93-44e0-932c-019ca6eb169d\Exec>xunit.console.exe Microsoft.Win32.SystemEvents.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=nonnetfxtests -notrait category=nonwindowstests -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: Microsoft.Win32.SystemEvents.Tests (app domain = on [no shadow copy], method display = ClassAndMethod, method display options = None)
  Discovered:  Microsoft.Win32.SystemEvents.Tests (found 26 of 27 test cases)
  Starting:    Microsoft.Win32.SystemEvents.Tests (parallel test collections = on, max threads = 2)
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:02:11
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:04:12
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:06:12
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:08:12
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:10:12
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:12:12
   Microsoft.Win32.SystemEvents.Tests: [Long Running Test] 'Microsoft.Win32.SystemEventsTests.InvokeOnEventsThreadTests.InvokeOnEventsThreadRunsAsynchronously', Elapsed: 00:14:12

[ViktorHofer]

Another failure on netcoreapp-Windows_NT-Release-x64-Windows.7.Amd64.Open: https://dnceng.visualstudio.com/public/_build/results?buildId=334246&view=ms.vss-test-web.build-test-results-tab&runId=9827812&paneView=debug

Microsoft.Win32.SystemEventsTests.CreateTimerTests.TimerElapsedSignaled

Assert.False() Failure\r\nExpected: False\r\nActual: True
   at Microsoft.Win32.SystemEventsTests.CreateTimerTests.TimerElapsedSignaled() in /_/src/Microsoft.Win32.SystemEvents/tests/SystemEvents.CreateTimer.cs:line 70

Will send a PR to disable the test.