Crash in wxAUI/wxOverlay on Mac when tabs/windows are dragged

[dkulp]

Description

With the recent changes to use wxOverlay, we're seeing a bunch of crashes and other issues when dragging/docking windows with wxAUI on the Mac. This can be seen with the wxAUI sample with a debug build of wxWidgets. If you start the sample and immediately try to drag one of the tabs off the notebook, an assert is raised and then the app crashes.

MacOS Sonoma 14.4, Xcode 15

[vadz]

Thanks, I can reproduce this and AFAICS this happens because m_overlayWindow is null, i.e. either wxOverlayImpl::CreateOverlayWindow() hadn't been called, somehow, or it failed. But I don't immediately see why would this happen...

cc @AliKet

[dkulp]

I BELIEVE this can be fixed via:

--- a/src/osx/cocoa/overlay.mm
+++ b/src/osx/cocoa/overlay.mm
@@ -280,7 +280,7 @@ void wxOverlayImpl::Clear(wxDC* dc)

 void wxOverlayImpl::Reset()
 {
-    if ( m_window )
+    if ( m_window && m_overlayWindow)
     {
         // erase whatever was drawn on the overlay the last time
         wxClientDC dc(m_window);

It seems to occur when you try to drag the tab (or AUI frame) over something that shouldn't accept the tab/frame in which case there wouldn't be an overlay for it. Thus, there would be nothing to clear/reset.

[vadz]

Thanks! I agree that Reset() shouldn't do anything in this case, so I'm pushing a commit with the same effect, with which I can't reproduce the bug any more. Please let me know if you still see any problems.