Crash when minimizing or alt-tabbing to the desktop

[ranseur]

Game version: 0.C-29754-g34eea34 (console) (build 7627, latest experimental)

Operating system: Windows 8.1 64bit

Tiles or curses: curses (but tiles still has problems with screen resizing)

Mods active: any

Expected behavior

No crash, minimizes/alt-tabs fine.

Actual behavior

Steps to reproduce the behavior

Launch the game then minimize the game or alt-tab to the desktop (alt-tabbing to other windows is fine).

Other notes

After the crash the terminal width and height returns to the defaults 80 and 24 regardless of what it was to before. (The game also crashes and behaves erratically when trying to resize the window)
The log doesn't produce anything useful.

[Rail-Runner]

Having same issue on Windows XP, build 7626, but only on version without tiles. Screen resize works good on both versions.

[ranseur]

You are right, tested again now and minimizing with the tiles version and it works. Screen resize is still pretty bad for me though. In the tiles version you can resize the window in the main menu and ingame, if you try during during character creation or while in the options menu it will blank the screen then crash. In the curses version you can resize the screen only in the main menu I think, ingame and in the submenus it blanks and crashes.

[DAOWAce]

Crashes when tabbing out of fullscreen as well. After the crash the size resets to default.

Borderless doesn't like to be minimized either; it tries to keep itself the active window and may also crash when attempting to minimize it.

The only way to play without it terminating is in regular windowed mode. I haven't had any real issues with resizing/minimizing there. I have had the screen blank on resize in the menu, but just activate a redraw (move selector) and it comes back fine.

This is for the Tiles version.
Windows 10 x64 1703. NVIDIA 1080 Ti with the latest 2 driver releases tested.

[Vzor-]

I get a crash if I resize or minimize the window on any screen other than the main menu.
Curses, Windows 10 64, 125% screen scaling. 0.C-7924 and 0.C-7979 (and likely others)

[Night-Pryanik]

Can't confirm on latest experimental as of 07 February 2019, Windows 10 Pro x64, tiles, neither in fullscreen nor windowed mode. Is this still an issue for anyone?

[DAOWAce]

Still crashes when tabbing out in fullscreen on build #8505

But upon further testing, it only appears to do this under the d3d9 renderer. I haven't gotten it to crash with d3d11 in my limited testing.

[Night-Pryanik]

Yeah, it seems that using d3d9 renderer indeed crashes the game.

[DSeyka]

Crashes for me as well, but only in direct3d fullscreen mode, and only when alt-tabbing. Both windowed modes work fine. Tiles 8505, Win10.
Attaching stacktrace since it looks slightly different from what DAOWAce has provided:

[neitsa]

I took a few notes while debugging the problem. I don't have a fix... yet :)

Ok, please bear with me this is a bit complex.

Crash Repro

• Be on Windows :)
• catacysm-tiles.exe
• Go to options > Graphics
  • Set renderer to DirectX
  • Set Windows to fullscreen

Restart the game; wait for the window to appear; then ALT-TAB: crash!

Crash analysis

First you need to compile SDL2 in debug mode to get all the symbols.

Here the stack trace at time of crash:

 	d3d9.dll!CD3DHal::SetTransformI()	Unknown
 	d3d9.dll!CD3DBase::SetTransform()	Unknown
>	SDL2.dll!D3D_UpdateViewport(SDL_Renderer * renderer) Line 1112	C
 	SDL2.dll!SDL_RenderSetViewport_REAL(SDL_Renderer * renderer, const SDL_Rect * rect) Line 1470	C
 	SDL2.dll!UpdateLogicalSize(SDL_Renderer * renderer) Line 1396	C
 	SDL2.dll!SDL_RendererEventWatch(void * userdata, SDL_Event * event) Line 166	C
 	SDL2.dll!SDL_PushEvent_REAL(SDL_Event * event) Line 740	C
 	SDL2.dll!SDL_SendWindowEvent(SDL_Window * window, unsigned char windowevent, int data1, int data2) Line 198	C
 	SDL2.dll!SDL_OnWindowResized(SDL_Window * window) Line 2582	C
 	SDL2.dll!SDL_SendWindowEvent(SDL_Window * window, unsigned char windowevent, int data1, int data2) Line 125	C
 	SDL2.dll!WIN_WindowProc(HWND__ * hwnd, unsigned int msg, unsigned __int64 wParam, __int64 lParam) Line 824	C
 	[External Code]	
 	SDL2.dll!D3D_Reset(SDL_Renderer * renderer) Line 355	C
 	SDL2.dll!D3D_ActivateRenderer(SDL_Renderer * renderer) Line 412	C
 	SDL2.dll!D3D_SetRenderTarget(SDL_Renderer * renderer, SDL_Texture * texture) Line 1068	C
 	SDL2.dll!SDL_SetRenderTarget_REAL(SDL_Renderer * renderer, SDL_Texture * texture) Line 1248	C
 	SDL2.dll!SDL_SetRenderTarget(SDL_Renderer * a, SDL_Texture * b) Line 351	C
 	Cataclysm.exe!SetRenderTarget(const std::unique_ptr<SDL_Renderer,SDL_Renderer_deleter> & renderer, const std::unique_ptr<SDL_Texture,SDL_Texture_deleter> & texture) Line 134	C++
 	Cataclysm.exe!refresh_display() Line 795	C++
 	Cataclysm.exe!try_sdl_update() Line 820	C++
 	Cataclysm.exe!CheckMessages() Line 3104	C++
 	Cataclysm.exe!input_manager::get_input_event() Line 3541	C++
 	Cataclysm.exe!input_context::handle_input(int timeout) Line 791	C++
 	Cataclysm.exe!main_menu::handle_input_timeout(input_context & ctxt) Line 180	C++
 	Cataclysm.exe!main_menu::opening_screen() Line 418	C++
 	Cataclysm.exe!SDL_main(int argc, char * * argv) Line 677	C++
 	Cataclysm.exe!main_getcmdline(...) Line 175	C
 	Cataclysm.exe!WinMain(HINSTANCE__ * hInst, HINSTANCE__ * hPrev, char * szCmdLine, int sw) Line 205	C
 	[External Code]	

Since we obviously don't have the DirextX source code, here's the assembly location (in d3d9.dll!CD3DHal::SetTransformI):

00007FFBEF526877  mov         rcx,qword ptr [rbx+4118h]  ; rcx is null
00007FFBEF52687E  mov         r8,rsi  
00007FFBEF526881  mov         edx,edi  
00007FFBEF526883  mov         rax,qword ptr [rcx] ; crash

So basically RBX is some kind of class or structure; the code tries to get a pointer at offset 0x4118 from that structure and dereference it (so this value is expected to be a pointer) but that pointer is NULL so we get a crash.

Checking what is RBX is pretty simple as the stack trace in D3D is small enough to backtrack visually. Moreover we have the source for SDL:

// in \SDL2-2.0.9\src\render\direct3d\SDL_render_d3d.c

static int
D3D_UpdateViewport(SDL_Renderer * renderer)
{
    D3D_RenderData *data = (D3D_RenderData *) renderer->driverdata;
    
    // [snip]

    IDirect3DDevice9_SetTransform(data->device, D3DTS_PROJECTION, &matrix);

    return 0;
}  

Here's the data->device address when IDirect3DDevice9_SetTransform is called:

> ? data->device
    0x0000021c771e75a0 {lpVtbl=0x0000021c771ebbc0 {QueryInterface=0x00007ffbef530a10 {d3d9.dll!CBaseDevice::QueryInterface(void)} ...} }	IDirect3DDevice9 *

Note: data->device is a IDirect3DDevice9 *.

Now, RBX at time of crash:

> ? rbx
0x0000021c771e75a0 // same address!

As you can see the RBX register is data->device, a IDirect3DDevice9 interface.

Sadly the big offset (0x4118) is way outside of the publicly defined source and symbols for the IDirect3DDevice9 interface...

So in these kind of cases, a good idea is usually to restart the program and see what actually changes this offset.

Going from the start

DirectX Device Initialization

The D3D_RenderData structure (for which device is a member) is allocated in D3D_CreateRenderer() in the SDL code:

SDL_Renderer *
D3D_CreateRenderer(SDL_Window * window, Uint32 flags)
{
    SDL_Renderer *renderer;
    D3D_RenderData *data;
    
    // ...

    data = (D3D_RenderData *) SDL_calloc(1, sizeof(*data));

And then the device is initialized in the same function using IDirect3D9_CreateDevice.

    result = IDirect3D9_CreateDevice(data->d3d, data->adapter,
                                     D3DDEVTYPE_HAL,
                                     pparams.hDeviceWindow,
                                     device_flags,
                                     &pparams, &data->device);

Now, just after the above line has executed we set a breakpoint to capture any overwrite of the data->device + 0x4118 location.

At some point after ALT-TABing the data-on-write breakpoint is triggered and leads to the following stack trace:

 	d3d9.dll!CD3DBase::Destroy()	Unknown
 	d3d9.dll!CD3DHal::Destroy()	Unknown
 	d3d9.dll!CBaseDevice::ResetMain(struct _D3DPRESENT_PARAMETERS_ *,struct D3DDISPLAYMODEEX const *)	Unknown
 	d3d9.dll!CBaseDevice::Reset(struct _D3DPRESENT_PARAMETERS_ *)	Unknown
 	SDL2.dll!D3D_Reset(SDL_Renderer * renderer) Line 355	C
 	SDL2.dll!D3D_ActivateRenderer(SDL_Renderer * renderer) Line 412	C
 	SDL2.dll!D3D_SetRenderTarget(SDL_Renderer * renderer, SDL_Texture * texture) Line 1068	C
 	SDL2.dll!SDL_SetRenderTarget_REAL(SDL_Renderer * renderer, SDL_Texture * texture) Line 1248	C
 	SDL2.dll!SDL_RendererEventWatch(void * userdata, SDL_Event * event) Line 164	C
 	SDL2.dll!SDL_PushEvent_REAL(SDL_Event * event) Line 740	C
 	SDL2.dll!SDL_SendWindowEvent(SDL_Window * window, unsigned char windowevent, int data1, int data2) Line 198	C
 	SDL2.dll!SDL_OnWindowResized(SDL_Window * window) Line 2582	C
 	SDL2.dll!SDL_SendWindowEvent(SDL_Window * window, unsigned char windowevent, int data1, int data2) Line 125	C
 	SDL2.dll!WIN_WindowProc(HWND__ * hwnd, unsigned int msg, unsigned __int64 wParam, __int64 lParam) Line 824	C
 	[External Code]	
 	SDL2.dll!WIN_SetWindowFullscreen(SDL_VideoDevice * _this, SDL_Window * window, SDL_VideoDisplay * display, SDL_bool fullscreen) Line 692	C
 	SDL2.dll!SDL_UpdateFullscreenMode(SDL_Window * window, SDL_bool fullscreen) Line 1338	C
 	SDL2.dll!SDL_MinimizeWindow_REAL(SDL_Window * window) Line 2223	C
 	SDL2.dll!SDL_OnWindowFocusLost(SDL_Window * window) Line 2666	C
 	SDL2.dll!SDL_SendWindowEvent(SDL_Window * window, unsigned char windowevent, int data1, int data2) Line 179	C
 	SDL2.dll!SDL_SetKeyboardFocus(SDL_Window * window) Line 655	C
 	SDL2.dll!WIN_WindowProc(HWND__ * hwnd, unsigned int msg, unsigned __int64 wParam, __int64 lParam) Line 479	C
 	[External Code]	
 	SDL2.dll!WIN_PumpEvents(SDL_VideoDevice * _this) Line 1072	C
 	SDL2.dll!SDL_PumpEvents_REAL() Line 657	C
 	SDL2.dll!SDL_WaitEventTimeout_REAL(SDL_Event * event, int timeout) Line 696	C
 	SDL2.dll!SDL_PollEvent_REAL(SDL_Event * event) Line 678	C
 	SDL2.dll!SDL_PollEvent(SDL_Event * a) Line 153	C
>	Cataclysm.exe!CheckMessages() Line 2711	C++
 	Cataclysm.exe!input_manager::get_input_event() Line 3541	C++
 	Cataclysm.exe!input_context::handle_input(int timeout) Line 791	C++
 	Cataclysm.exe!main_menu::handle_input_timeout(input_context & ctxt) Line 180	C++
 	Cataclysm.exe!main_menu::opening_screen() Line 418	C++
 	Cataclysm.exe!SDL_main(int argc, char * * argv) Line 677	C++
 	Cataclysm.exe!main_getcmdline(...) Line 175	C
 	Cataclysm.exe!WinMain(HINSTANCE__ * hInst, HINSTANCE__ * hPrev, char * szCmdLine, int sw) Line 205	C
 	[External Code]	

In d3d9.dll!CD3DBase::Destroy(), the code is setting our data->device + big_offset to 0:

00007FFBEF513957  mov         qword ptr [rbx+4118h],rsi ; note: RSI = 0

If we compare data (D3D_RenderData *data) before and after the reset we have:

Before:

> ? data
0x000001c06b1aa988 {d3dDLL=d3d9.dll!0x00007ffbef500000 d3d=0x000001c06a549020 <No type information available in symbol file for d3d9.dll> ...}
    d3dDLL: d3d9.dll!0x00007ffbef500000 (Type information missing from symbol file)
    d3d: 0x000001c06a549020 <No type information available in symbol file for d3d9.dll>
    device: 0x000001c06ae0b620 {lpVtbl=0x000001c06ae0fc40 {QueryInterface=0x00007ffbef530a10 {d3d9.dll!CBaseDevice::QueryInterface(void)} ...} }

With the big offset (we can see there is pointer there, so it's still present):

> dq 0x000001c06ae0b620 + 0x4118
0x000001C06AE0F738  000001c06adf6140   

And after (the content at that memory address is 0 just after calling reset):

> dq 0x000001c06ae0b620 + 0x4118
0x000001C06AE0F738  0000000000000000

From now on, all invocation on the renderer (that is every call made by SDL to directX) will fail with the following return code (from DirectX);

? result
0x8876086c

It happens this is the following code:

#define D3DERR_INVALIDCALL                      MAKE_D3DHRESULT(2156)

Which confirm that once the renderer is reset, there no point in continuing calling any API on it, simply because it's no more valid.

Root Cause (Somewhat...)

By reading the stack trace it is clear that the renderer is invalidated during the minimization of the Windows.

This is in the CDDA code:

//Check for any window messages (keypress, paint, mousemove, etc)
void CheckMessages()
{
    SDL_Event ev;
    bool quit = false;
    bool text_refresh = false;
    bool is_repeat = false;
    if( HandleDPad() ) {
        return;
    }

    last_input = input_event();
    while( SDL_PollEvent( &ev ) ) { // inside

A bit before the crash in SDL_RendererEventWatch(void *userdata, SDL_Event *event) (called from SDL_PollEvent()`:

> ? event->window
{type=0x00000200 timestamp=0x00022ef6 windowID=0x00000001 ...}
    type: 0x00000200
    timestamp: 0x00022ef6
    windowID: 0x00000001
    event: 0x06 '\x6'      // !!! message type !!!
    padding1: 0x00 '\0'
    padding2: 0x00 '\0'
    padding3: 0x00 '\0'
    data1: 0x00000780  // w
    data2: 0x00000430  // h

The event is type 0x6 (type = 0x200 indicates it's a window event) which simply means the window is changing its size:

SDL_WINDOWEVENT_SIZE_CHANGED // 6 

I did a bit of SDL logging by placing some breakpoints and here are the messages which transit into SDL before the crash:

17:51:40.224 INFO SDL : Window event: 4  // SDL_WINDOWEVENT_MOVED
17:51:40.226 INFO SDL : Window event: 6  // SDL_WINDOWEVENT_SIZE_CHANGED
17:51:40.226 INFO SDL : Window event: 5  // SDL_WINDOWEVENT_RESIZED
17:51:40.227 INFO SDL : Window event: 1  // SDL_WINDOWEVENT_SHOWN
17:51:40.227 INFO SDL : Window event: C  // SDL_WINDOWEVENT_FOCUS_GAINED
17:51:40.246 INFO SDL : Window event: 3  // SDL_WINDOWEVENT_EXPOSED
17:51:41.146 INFO SDL : Window event: A  // SDL_WINDOWEVENT_ENTER
18:34:04.761 INFO SDL : Window event: 4  // SDL_WINDOWEVENT_MOVED
18:34:04.766 INFO SDL : Window event: 6  // SDL_WINDOWEVENT_SIZE_CHANGED
18:34:04.785 INFO SDL : Window event: 7  // SDL_WINDOWEVENT_MINIMIZED
18:34:04.823 INFO SDL : Window event: D  // SDL_WINDOWEVENT_FOCUS_LOST
18:34:04.824 INFO SDL : Window event: 3  // SDL_WINDOWEVENT_EXPOSED    

It's pretty clear that when we are in fullscreen and we get out of the window, it is minimized and the renderer is invalidated.

At this point I don't know if it is a normal behavior or not...

(to be continued)

[Alicecomma]

Just chiming in:

Settings > Graphics > Renderer = x

• x = direct3d: crashes every time on alt-tab.
• x = direct3d11: no crash, but focus is regained only after key-press and only part of the game is rendered.
• x = opengl: no crash, but focus regains slowly (~5 seconds) - first, graphics are rendered after 2 sec and second the control is regained after 3 sec.
• x = software: no crash, same behaviour as direct3d11.