Deadlock issue with webview2 on shutdown

[pankajpalOlk]

Hi, I am using webview2 nuget package in outlook for actionable messages scenario. Of lately, a lot of outlook users reported application hang issue with this signature:

"deadlock_cfffffff_embeddedbrowserwebview.dll!base::internal::invoker_base::internal::bindstate__lambda_at_.._.._edge_embedded_browser_client_win_deprecated430_web_resource_response.cc:135:13_,base::value,atl::ccomgitptr_istream___,base::value_"

I debugged through a lot of core files and here are my findings:

We are using add_WebResourceRequested() to intercept http requests and return an IStream (created via SHCreateStreamOnFileW()).

This will internally call a lambda from BuildCdpRequestAsync() method on EBWorkerThread which will run in a different thread than the main UI thread. Assume if someone closes the webview at this point in time by triggering EmbeddedBrowserWebView::~EmbeddedBrowserWebView() on main thread. This will result in main thread doing the cleanup as part of this dtor() and internally try to clean up the unique_ptr<EBWorkerThread> and in the process the main thread starts waiting on the worker thread.

Meanwhile, worker thread executes the delegates, but while executing the lamdba, it calls combase!ClassicSTAThreadDispatchCrossApartmentCall for multiple reasons - destroying local ISteam objects, IStream::CopyTo() etc. This STA dispatch call means the worker thread is going to wait for Main thread to finish its job. And main thread is already waiting on this worker thread to join to itself. Hence a deadlock.

Is this a known issue? If it helps, here is how we are using webview2, If this is not a standard way to use request interception then please suggest alternatives:
https://office.visualstudio.com/OC/_git/OutlookOnyxHost?path=%2Fsrc%2Fwebview2%2FWebViewInstance.cpp&version=GBmaster&line=1318&lineEnd=1318&lineStartColumn=18&lineEndColumn=30&lineStyle=plain

PS: We are using embedded_browser_webview_deprecate430 right now and are in the process of upgrading to the latest nuget package.

Do let me know what further information is required from my end.. Here is a watson bucket for such failures:
https://watsonportal.microsoft.com/Failure?FailureSearchText=410d4a78-fe5d-bbc9-f62c-6467498b6fc1

Here is the top of stack trace for both the involved threads (only the webview related parts):

Main thread:
0:000> k10

Child-SP RetAddr Call Site

00 00000087632fe2f8 00007fff40ab8b03 ntdll!ZwWaitForSingleObject+0x14 [minkernel\ntdll\daytona\objfre\amd64\usrstubs.asm @ 211]
01 00000087632fe300 00007fff042d8a43 KERNELBASE!WaitForSingleObjectEx+0x93 [minkernel\kernelbase\synch.c @ 1328]
02 00000087632fe3a0 00007fff042cab15 EmbeddedBrowserWebView!base::PlatformThread::Join+0xc3 [D:\g\Edge\src\base\threading\platform_thread_win.cc @ 313]
03 (Inline Function) ---------------- EmbeddedBrowserWebView!base::Thread::Stop+0x22 [D:\g\Edge\src\base\threading\thread.cc @ 259] 04 00000087632fe480 00007fff041e1354 EmbeddedBrowserWebView!base::Thread::~Thread+0x35 [D:\g\Edge\src\base\threading\thread.cc @ 128] 05 (Inline Function) ---------------- EmbeddedBrowserWebView!std::__1::default_deletebase::Thread::operator()+0x8 [D:\g\Edge\src\buildtools\third_party\libc++\trunk\include\memory @ 2378]
06 00000087632fe4c0 00007fff0425c189 EmbeddedBrowserWebView!std::__1::unique_ptr<base::Thread,std::__1::default_deletebase::Thread >::reset+0x1e [D:\g\Edge\src\buildtools\third_party\libc++\trunk\include\memory @ 2634]
07 (Inline Function) ---------------- EmbeddedBrowserWebView!std::__1::unique_ptr<base::Thread,std::__1::default_delete<base::Thread> >::~unique_ptr+0x7 [D:\g\Edge\src\buildtools\third_party\libc++\trunk\include\memory @ 2587] 08 00000087632fe4f0 00007fff04264fca EmbeddedBrowserWebView!embedded_browser_webview_deprecated430::EmbeddedBrowserWebView::~EmbeddedBrowserWebView+0x141 [D:\g\Edge\src\edge_embedded_browser\client\win\deprecated430\embedded_browser_webview_impl.cc @ 210] 09 00000087632fe540 00007fff04264fa0 EmbeddedBrowserWebView!embedded_browser_webview_deprecated430::EmbeddedBrowserWebView::~EmbeddedBrowserWebView+0x10 [D:\g\Edge\src\edge_embedded_browser\client\win\deprecated430\embedded_browser_webview_impl.cc @ 205] 0a 00000087632fe580 00007fff044fd045 EmbeddedBrowserWebView!Microsoft::WRL::Details::RuntimeClassImpl<Microsoft::WRL::RuntimeClassFlags<2>,1,0,1,ICoreWebView2Deprecated430,ICoreWebView2Deprecated430Staging,ICoreWebView2Deprecated430Internal>::Release+0x26 [D:\g\Edge\src\third_party\depot_tools\win_toolchain\vs_files\f32a5438c50220c804b81cde79b151e5ac6630c4\win_sdk\Include\10.0.18362.0\winrt\wrl\implements.h @ 1590] 0b (Inline Function) ---------------- OutlookWebHost!Microsoft::WRL::ComPtr::InternalRelease+0x20 [c:\program files (x86)\windows kits\10\include\10.0.14393.0\winrt\wrl\client.h @ 226]
0c (Inline Function) ---------------- OutlookWebHost!Microsoft::WRL::ComPtr<ICoreWebView2>::operator=+0x20 [c:\program files (x86)\windows kits\10\include\10.0.14393.0\winrt\wrl\client.h @ 288] 0d 00000087632fe5b0 00007fff04502ceb OutlookWebHost!Outlook::WebHost::Details::WebViewInstance::CloseWebView+0x1b5 [d:\a\1\s\src\webview2\webviewinstance.cpp @ 1597] 0e 00000087632fe5f0 00007fff0451f64a OutlookWebHost!Outlook::WebHost::Details::WebViewInstance::Shutdown+0x5b [d:\a\1\s\src\webview2\webviewinstance.cpp @ 1530] 0f 00000087632fe660 00007ff7`a5e6f893 OutlookWebHost!Outlook::WebHost::Details::ActionableMessagesInstance::Shutdown+0x1fa [d:\a\1\s\src\webview2\actionablemessagesinstance.cpp @ 568]

EBWorker thread:

Child-SP RetAddr Call Site

00 000000876807e228 00007fff431e1cad win32u!ZwUserMsgWaitForMultipleObjectsEx+0x14 [windows\core\umode\daytona\objfre\amd64\usrstubs.asm @ 8990]
01 000000876807e230 00007fff427949b6 user32!RealMsgWaitForMultipleObjectsEx+0x1d [windows\core\ntuser\client\daytona\objfre\amd64\client.cxx @ 1896]
02 000000876807e270 00007fff42798025 combase!CCliModalLoop::BlockFn+0x186 [onecore\com\combase\dcomrem\callctrl.cxx @ 2233]
03 000000876807e330 00007fff4279810b combase!ModalLoop+0x9d [onecore\com\combase\dcomrem\chancont.cxx @ 166]
04 000000876807e3a0 00007fff42752327 combase!ClassicSTAThreadDispatchCrossApartmentCall+0x5b [onecore\com\combase\dcomrem\chancont.cxx @ 321]
05 (Inline Function) ---------------- combase!CSyncClientCall::SwitchAptAndDispatchCall+0x325 [onecore\com\combase\dcomrem\channelb.cxx @ 5696] 06 000000876807e3e0 00007fff427951f3 combase!CSyncClientCall::SendReceive2+0x407 [onecore\com\combase\dcomrem\channelb.cxx @ 5377] 07 (Inline Function) ---------------- combase!SyncClientCallRetryContext::SendReceiveWithRetry+0x24 [onecore\com\combase\dcomrem\callctrl.cxx @ 1617]
08 (Inline Function) ---------------- combase!CSyncClientCall::SendReceiveInRetryContext+0x24 [onecore\com\combase\dcomrem\callctrl.cxx @ 567] 09 000000876807e5e0 00007fff42753138 combase!ClassicSTAThreadSendReceive+0xa3 [onecore\com\combase\dcomrem\callctrl.cxx @ 549] 0a 000000876807e710 00007fff427976b4 combase!CSyncClientCall::SendReceive+0x128 [onecore\com\combase\dcomrem\ctxchnl.cxx @ 783] 0b 000000876807e940 00007fff427be3ae combase!CClientChannel::SendReceive+0x84 [onecore\com\combase\dcomrem\ctxchnl.cxx @ 653] 0c 000000876807e9b0 00007fff43738b95 combase!NdrExtpProxySendReceive+0x4e [onecore\com\combase\ndr\ndrole\proxy.cxx @ 2002] 0d (Inline Function) ---------------- rpcrt4!Ndr64pSendReceive+0x2f [minio\rpc\ndr64\cltcall.cxx @ 184]
0e 000000876807e9e0 00007fff427bc41b rpcrt4!NdrpClientCall3+0x395 [minio\rpc\ndr64\cltcall.cxx @ 328]
0f 000000876807ed40 00007fff4283d192 combase!ObjectStublessClient+0x13b [onecore\com\combase\ndr\ndrole\amd64\stblsclt.cxx @ 369]
10 000000876807f0d0 00007fff42797409 combase!ObjectStubless+0x42 [onecore\com\combase\ndr\ndrole\amd64\stubless.asm @ 176]
11 (Inline Function) ---------------- combase!RemoteReleaseRifRefHelper+0x122 [onecore\com\combase\dcomrem\marshal.cxx @ 8446] 12 000000876807f120 00007fff42796e86 combase!RemoteReleaseRifRef+0x219 [onecore\com\combase\dcomrem\marshal.cxx @ 8353] 13 000000876807f1e0 00007fff42779c26 combase!CStdMarshal::DisconnectCliIPIDs+0x596 [onecore\com\combase\dcomrem\marshal.cxx @ 5025] 14 000000876807f3b0 00007fff427711d3 combase!CStdMarshal::DisconnectWorker_ReleasesLock+0x566 [onecore\com\combase\dcomrem\marshal.cxx @ 4366] 15 (Inline Function) ---------------- combase!CStdIdentity::{dtor}+0x9b [onecore\com\combase\dcomrem\stdid.cxx @ 396]
16 000000876807f4d0 00007fff0426ff1a combase!CStdIdentity::CInternalUnk::Release+0x2b3 [onecore\com\combase\dcomrem\stdid.cxx @ 944]
17 (Inline Function) ---------------- EmbeddedBrowserWebView!Microsoft::WRL::ComPtr<IStream>::InternalRelease+0x1f [D:\g\Edge\src\third_party\depot_tools\win_toolchain\vs_files\f32a5438c50220c804b81cde79b151e5ac6630c4\win_sdk\Include\10.0.18362.0\winrt\wrl\client.h @ 176] 18 (Inline Function) ---------------- EmbeddedBrowserWebView!Microsoft::WRL::ComPtr::~ComPtr+0x1f [D:\g\Edge\src\third_party\depot_tools\win_toolchain\vs_files\f32a5438c50220c804b81cde79b151e5ac6630c4\win_sdk\Include\10.0.18362.0\winrt\wrl\client.h @ 231]
19 (Inline Function) ---------------- EmbeddedBrowserWebView!embedded_browser_webview_deprecated430::internal::WebResourceResponse::BuildCdpRequestAsync::<unnamed-tag>::operator()+0x214 [D:\g\Edge\src\edge_embedded_browser\client\win\deprecated430\web_resource_response.cc @ 152] 1a (Inline Function) ---------------- EmbeddedBrowserWebView!base::internal::FunctorTraits<lambda at ../../edge_embedded_browser/client/win/deprecated430/web_resource_response.cc:135:13',void>::Invoke+0x23f [D:\g\Edge\src\base\bind_internal.h @ 393] 1b (Inline Function) ---------------- EmbeddedBrowserWebView!base::internal::InvokeHelper<0,base::Value>::MakeItSo+0x23f [D:\g\Edge\src\base\bind_internal.h @ 651]
1c (Inline Function) ---------------- EmbeddedBrowserWebView!base::internal::Invoker<base::internal::BindState<lambda at ../../edge_embedded_browser/client/win/deprecated430/web_resource_response.cc:135:13',base::Value,ATL::CComGITPtr >,base::Value ()>::RunImpl+0x23f [D:\g\Edge\src\base\bind_internal.h @ 724]
1d 000000876807f510 00007fff0421cdca EmbeddedBrowserWebView!base::internal::Invoker<base::internal::BindState<lambda at ../../edge_embedded_browser/client/win/deprecated430/web_resource_response.cc:135:13',base::Value,ATL::CComGITPtr<IStream> >,base::Value ()>::RunOnce+0x266 [D:\g\Edge\src\base\bind_internal.h @ 693] 1e (Inline Function) ---------------- EmbeddedBrowserWebView!base::OnceCallback<base::Value ()>::Run+0x18 [D:\g\Edge\src\base\callback.h @ 99]
1f 000000876807f710 00007fff0421ce43 EmbeddedBrowserWebView!base::internal::ReturnAsParamAdapterbase::Value+0x43 [D:\g\Edge\src\base\post_task_and_reply_with_result_internal.h @ 22]

AB#27452601

[champnic]

This is fixed in Edge 86.0.573.0+. Thanks!