Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure System.Threading.ThreadPools.Tests.ThreadPoolTests.FileStreamFlushAsyncThreadPoolDeadlockTest #89702

Closed
v-wenyuxu opened this issue Jul 31, 2023 · 8 comments
Closed

Test failure System.Threading.ThreadPools.Tests.ThreadPoolTests.FileStreamFlushAsyncThreadPoolDeadlockTest #89702

v-wenyuxu opened this issue Jul 31, 2023 · 8 comments
Assignees
@kunalspathak
Labels
arch-x86 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI JitStress CLR JIT issues involving JIT internal stress modes os-linux Linux OS (any supported distro) os-windows
Milestone
8.0.0

Comments

@v-wenyuxu
Copy link

Failed in: runtime-coreclr libraries-jitstress 20230730.1

Failed tests:

net8.0-linux-Release-arm-CoreCLR_checked-no_tiered_compilation-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - System.Threading.ThreadPools.Tests.ThreadPoolTests.FileStreamFlushAsyncThreadPoolDeadlockTest
net8.0-windows-Release-x86-CoreCLR_checked-zapdisable-Windows.10.Amd64.Open
    - BasicEventSourceTests.TestsManifestGeneration.Test_EventSource_EtwManifestGenerationRollover

Error message:

Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Half-way through waiting for remote process.
Timed out at 7/30/2023 7:58:09 AM after 60000ms waiting for remote process.
	Process ID: 307
	Handle: 1412
	Name: dotnet
	MainModule: /root/helix/work/correlation/dotnet
	StartTime: 7/30/2023 7:57:09 AM
	TotalProcessorTime: 00:00:05.1800000

Stack trace:

   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 225
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs:line 36
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 57
@v-wenyuxu v-wenyuxu added arch-x86 os-linux Linux OS (any supported distro) os-windows JitStress CLR JIT issues involving JIT internal stress modes labels Jul 31, 2023
@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jul 31, 2023
@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jul 31, 2023
@jeffschwMSFT jeffschwMSFT added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 31, 2023
@ghost
Copy link

ghost commented Jul 31, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Failed in: runtime-coreclr libraries-jitstress 20230730.1

Failed tests:

net8.0-linux-Release-arm-CoreCLR_checked-no_tiered_compilation-(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7
    - System.Threading.ThreadPools.Tests.ThreadPoolTests.FileStreamFlushAsyncThreadPoolDeadlockTest
net8.0-windows-Release-x86-CoreCLR_checked-zapdisable-Windows.10.Amd64.Open
    - BasicEventSourceTests.TestsManifestGeneration.Test_EventSource_EtwManifestGenerationRollover

Error message:

Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Half-way through waiting for remote process.
Timed out at 7/30/2023 7:58:09 AM after 60000ms waiting for remote process.
	Process ID: 307
	Handle: 1412
	Name: dotnet
	MainModule: /root/helix/work/correlation/dotnet
	StartTime: 7/30/2023 7:57:09 AM
	TotalProcessorTime: 00:00:05.1800000

Stack trace:

   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 225
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs:line 36
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 57
Author: v-wenyuxu
Assignees: -
Labels:

arch-x86, os-linux, os-windows, JitStress, area-CodeGen-coreclr, untriaged, needs-area-label
Milestone: -

@vcsjones vcsjones removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Aug 2, 2023
@JulieLeeMSFT
Copy link
Member

@kunalspathak, PTAL.

@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone Aug 2, 2023
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Aug 2, 2023
@kunalspathak
Copy link
Member

I haven't seen this test failing since the run mentioned above. Closing it as flaky.

@v-wenyuxu v-wenyuxu reopened this Aug 17, 2023
@v-wenyuxu
Copy link
Author

Failed again in: runtime-coreclr libraries-jitstress 20230816.1

Failed tests:

net8.0-windows-Release-x86-CoreCLR_checked-zapdisable-Windows.10.Amd64.Open
    - BasicEventSourceTests.TestsManifestGeneration.Test_EventSource_EtwManifestGenerationRollover

Error message:

Microsoft.DotNet.RemoteExecutor.RemoteExecutionException : Half-way through waiting for remote process.
Memory load: 16

Image Name                     PID Services                                    
========================= ======== ============================================
svchost.exe                    756 BrokerInfrastructure, DcomLaunch, LSM,      
                                   PlugPlay, Power, SystemEventsBroker         
svchost.exe                    816 RpcEptMapper, RpcSs                         
svchost.exe                    964 TermService                                 
svchost.exe                    972 Appinfo, CertPropSvc, DsmSvc, gpsvc,        
                                   iphlpsvc, lfsvc, ProfSvc, sacsvr, Schedule, 
                                   SENS, SessionEnv, ShellHWDetection, Themes, 
                                   UserManager, Winmgmt, WpnService, wuauserv  
svchost.exe                    476 Dhcp, EventLog, lmhosts, TimeBrokerSvc,     
                                   vmictimesync                                
svchost.exe                    444 NcbService, PcaSvc, TrkWks, UALSVC,         
                                   UmRdpService, vmickvpexchange,              
                                   vmicshutdown, WdiSystemHost                 
svchost.exe                   1084 CDPSvc, EventSystem, FontCache, netprofm,   
                                   nsi, W32Time, WinHttpAutoProxySvc           
svchost.exe                   1092 BFE, CoreMessagingRegistrar, DPS, MpsSvc,   
                                   pla                                         
svchost.exe                   1164 vmicheartbeat                               
svchost.exe                   1228 CryptSvc, Dnscache, LanmanWorkstation,      
                                   NlaSvc, WinRM                               
svchost.exe                   1508 Wcmsvc                                      
svchost.exe                   1372 DiagTrack                                   
svchost.exe                   1408 AppHostSvc                                  
svchost.exe                   2096 StateRepository, tiledatamodelsvc           
svchost.exe                   2152 W3SVC, WAS                                  
svchost.exe                   2160 LanmanServer                                
svchost.exe                   3852 CDPUserSvc_52c29, OneSyncSvc_52c29          
svchost.exe                   2112 AppXSvc                                     
Timed out at 8/16/2023 8:21:29 AM after 60000ms waiting for remote process.
Wrote mini dump to: C:\h\w\A1A8095F\w\B747098A\uploads\3920.em1jdkok.srh.dmp
	Process ID: 3920
	Handle: 1172
	Name: dotnet
	MainModule: C:\h\w\A1A8095F\p\dotnet.exe
	StartTime: 8/16/2023 8:20:28 AM
	TotalProcessorTime: 00:00:07.0468750
	Threads:
		Thread #1 (OS 0x7BC)   [MTA]
			[HelperMethodFrame] (System.Threading.Thread.SleepInternal)
			System.Threading.Thread.Sleep(Int32)
			BasicEventSourceTests.TestsManifestGeneration.<Test_EventSource_EtwManifestGenerationRollover>b__3_0()
			[HelperMethodFrame_PROTECTOBJ] (System.RuntimeMethodHandle.InvokeMethod)
			System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(System.Object, IntPtr*)
			System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(System.Object, System.Reflection.BindingFlags)
			System.Reflection.RuntimeMethodInfo.Invoke(System.Object, System.Reflection.BindingFlags, System.Reflection.Binder, System.Object[], System.Globalization.CultureInfo)
			Microsoft.DotNet.RemoteExecutor.Program.Main(System.String[])
			[InlinedCallFrame]
		Thread #2 (OS 0x590) [Finalizer] [Background] 
			[DebuggerU2MCatchHandlerFrame]

Stack trace:

   at Microsoft.DotNet.RemoteExecutor.RemoteInvokeHandle.Dispose(Boolean disposing) in /_/src/Microsoft.DotNet.RemoteExecutor/src/RemoteInvokeHandle.cs:line 225
   at BasicEventSourceTests.TestsManifestGeneration.Test_EventSource_EtwManifestGenerationRollover() in /_/src/libraries/System.Diagnostics.Tracing/tests/BasicEventSourceTest/TestsManifestGeneration.Etw.cs:line 81
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args) in /_/src/coreclr/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.CoreCLR.cs:line 36
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr) in /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodBaseInvoker.cs:line 57

@JulieLeeMSFT
Copy link
Member

@mangod9, can your team take a look?

@mangod9
Copy link
Member

mangod9 commented Aug 17, 2023

The latest stack trace appears to be Etw Tracing related. @davmason so looking into something similar. Not sure if these are related. The original issue seems to be different though

@JulieLeeMSFT
Copy link
Member

Ping @davmason.

@davmason
Copy link
Member

The second failure is unrelated to the original issue, it is a dupe of #88027. Closing this issue

@davmason davmason mentioned this issue Aug 23, 2023
Open
@ghost ghost locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@kunalspathak kunalspathak

Labels
arch-x86 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI JitStress CLR JIT issues involving JIT internal stress modes os-linux Linux OS (any supported distro) os-windows
Projects
None yet
Milestone
8.0.0
Development

No branches or pull requests
7 participants
@vcsjones @jeffschwMSFT @davmason @kunalspathak @mangod9 @JulieLeeMSFT @v-wenyuxu