Open
Description
Describe the bug
System hang by XDP's Rebind tests.
Affected OS
- Windows
- Linux
- macOS
- Other (specify below)
Additional OS information
Hyper-V Windows VM created by "Quick Create"
MsQuic version
main
Steps taken to reproduce bug
pwsh .\scripts\prepare-machine.ps1 -UseXdp
pwsh .\scripts\build.ps1
.\artifacts\bin\windows\x64_Debug_schannel\msquictest.exe --gtest_filter="*RebindAddr*" --duoNic --gtest_repeat=1000Expected behavior
Pass test without issue.
Actual outcome
- Randomly system hang when run without WinDbg. VM disconnected. need reboot (cannot take log)
- Hit assert when running with WinDbg
Additional details
SendFlags = 0x400 is QUIC_CONN_SEND_FLAG_PATH_CHALLENGE
The worker thread's callstack
0:005> k
# Child-SP RetAddr Call Site
00 00000006`0a8ff000 00007ff8`c19ae60c msquic!QuicSendWriteFrames+0x1587 [C:\Users\User\workspace\msquic\src\core\send.c @ 956]
01 00000006`0a8ff320 00007ff8`c19783c0 msquic!QuicSendFlush+0x67c [C:\Users\User\workspace\msquic\src\core\send.c @ 1355]
02 00000006`0a8ff930 00007ff8`c199888f msquic!QuicConnDrainOperations+0x3b0 [C:\Users\User\workspace\msquic\src\core\connection.c @ 7662]
03 00000006`0a8ff9a0 00007ff8`c19972db msquic!QuicWorkerProcessConnection+0x24f [C:\Users\User\workspace\msquic\src\core\worker.c @ 578]
04 00000006`0a8ffb30 00007ff8`c19e0ef0 msquic!QuicWorkerLoop+0x13b [C:\Users\User\workspace\msquic\src\core\worker.c @ 741]
05 00000006`0a8ffba0 00007ff8`c19e09a9 msquic!CxPlatRunExecutionContexts+0xf0 [C:\Users\User\workspace\msquic\src\platform\platform_worker.c @ 481]
06 00000006`0a8ffc10 00007ff9`1bf6259d msquic!CxPlatWorkerThread+0x1b9 [C:\Users\User\workspace\msquic\src\platform\platform_worker.c @ 580]
07 00000006`0a8ffdf0 00007ff9`1d1caf38 KERNEL32!BaseThreadInitThunk+0x1d
08 00000006`0a8ffe20 00000000`00000000 ntdll!RtlUserThreadStart+0x28
Main thread is consistently same upto CxPlatTlsResetSchannel.
0:000> k
# Child-SP RetAddr Call Site
00 0000007f`0b92e8b8 00007ff9`1a58928f ntdll!NtClose+0x14
01 0000007f`0b92e8c0 00007ff9`192ddf46 KERNELBASE!CloseHandle+0x4f
02 0000007f`0b92e8f0 00007ff9`192ddf7c schannel!CEvent::~CEvent+0x1e
03 0000007f`0b92e920 00007ff9`192ddafe schannel!CSslUserContext::~CSslUserContext+0x24
04 0000007f`0b92e950 00007ff9`192e0441 schannel!CSslUserContext::`vector deleting destructor'+0xe
05 0000007f`0b92e980 00007ff9`192cf055 schannel!DeleteUserContext+0x41
06 0000007f`0b92e9b0 00007ff9`192e0cce schannel!SslDeleteUserContext+0x1d
07 0000007f`0b92e9e0 00007ff9`196d3c37 schannel!SpDeleteUserModeContext+0x2e
08 0000007f`0b92ea10 00007ff9`196db11b SspiCli!LsaDeleteSecurityContext+0x67
09 0000007f`0b92ea50 00007ff8`c2296834 SspiCli!DeleteSecurityContext+0x8b
0a 0000007f`0b92eaa0 00007ff8`c2295701 msquic!CxPlatTlsResetSchannel+0x34 [C:\Users\User\workspace\msquic\src\platform\tls_schannel.c @ 1638]
0b 0000007f`0b92ead0 00007ff8`c2249a1d msquic!CxPlatTlsUninitialize+0xb1 [C:\Users\User\workspace\msquic\src\platform\tls_schannel.c @ 1656]
0c 0000007f`0b92ebb0 00007ff8`c2226fd4 msquic!QuicCryptoUninitialize+0x9d [C:\Users\User\workspace\msquic\src\core\crypto.c @ 244]
0d 0000007f`0b92ebf0 00007ff8`c2217591 msquic!QuicConnFree+0x224 [C:\Users\User\workspace\msquic\src\core\connection.c @ 324]
0e 0000007f`0b92ec80 00007ff8`c221c9b0 msquic!QuicConnRelease+0x251 [C:\Users\User\workspace\msquic\src\core\connection.h @ 1095]
0f 0000007f`0b92ece0 00007ff6`1965aa61 msquic!MsQuicConnectionClose+0x470 [C:\Users\User\workspace\msquic\src\core\api.c @ 181]
10 0000007f`0b92ee20 00007ff6`19659ed4 msquictest!MsQuicConnection::Close+0x51 [C:\Users\User\workspace\msquic\src\inc\msquic.hpp @ 1043]
11 0000007f`0b92ee70 00007ff6`1964e4ce msquictest!MsQuicConnection::~MsQuicConnection+0x14 [C:\Users\User\workspace\msquic\src\inc\msquic.hpp @ 1022]
12 0000007f`0b92eeb0 00007ff6`194ed95e msquictest!QuicTestNatAddrRebind+0xb6e [C:\Users\User\workspace\msquic\src\test\lib\HandshakeTest.cpp @ 572]
13 0000007f`0b92f380 00007ff6`1972b9ae msquictest!WithFamilyArgs_RebindDatapathAddr_Test::TestBody+0x6e [C:\Users\User\workspace\msquic\src\test\bin\quic_gtest.cpp @ 1629]
14 0000007f`0b92f3d0 00007ff6`1972b653 msquictest!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test,void>+0x2e [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 2646]
15 0000007f`0b92f420 00007ff6`19702f1e msquictest!testing::internal::HandleExceptionsInMethodIfSupported<testing::Test,void>+0x73 [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 2689]
16 0000007f`0b92f500 00007ff6`19703c26 msquictest!testing::Test::Run+0xce [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 2728]
17 0000007f`0b92f570 00007ff6`19704735 msquictest!testing::TestInfo::Run+0x186 [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 2874]
18 0000007f`0b92f620 00007ff6`1970bfdd msquictest!testing::TestSuite::Run+0x2e5 [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 3052]
19 0000007f`0b92f750 00007ff6`1972ba6e msquictest!testing::internal::UnitTestImpl::RunAllTests+0x51d [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 5982]
1a 0000007f`0b92f950 00007ff6`1972b903 msquictest!testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>+0x2e [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 2646]
1b 0000007f`0b92f9a0 00007ff6`197050cf msquictest!testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl,bool>+0x73 [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 2689]
1c 0000007f`0b92fa80 00007ff6`195d7223 msquictest!testing::UnitTest::Run+0x16f [C:\Users\User\workspace\msquic\submodules\googletest\googletest\src\gtest.cc @ 5561]
1d 0000007f`0b92fb30 00007ff6`194f829e msquictest!RUN_ALL_TESTS+0x13 [C:\Users\User\workspace\msquic\submodules\googletest\googletest\include\gtest\gtest.h @ 2334]
1e 0000007f`0b92fb60 00007ff6`197c9b79 msquictest!main+0x24e [C:\Users\User\workspace\msquic\src\test\bin\quic_gtest.cpp @ 2639]
1f 0000007f`0b92fbd0 00007ff6`197c9ac2 msquictest!invoke_main+0x39 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 79]
20 0000007f`0b92fc20 00007ff6`197c997e msquictest!__scrt_common_main_seh+0x132 [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 288]
21 0000007f`0b92fc90 00007ff6`197c9bee msquictest!__scrt_common_main+0xe [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl @ 331]
22 0000007f`0b92fcc0 00007ff9`1bf6259d msquictest!mainCRTStartup+0xe [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp @ 17]
23 0000007f`0b92fcf0 00007ff9`1d1caf38 KERNEL32!BaseThreadInitThunk+0x1d
24 0000007f`0b92fd20 00000000`00000000 ntdll!RtlUserThreadStart+0x28