x32dbg hangs itself and the debugee

[Maktm]

Debugger version:
x32dbg v25 (Compiled on: Feb 28 2017, 05:07:48)

Operating system version and Service Pack (including 32 or 64 bits):
Windows 7 x64 Service Pack 1

Brief description of the issue:

When setting a breakpoint on a function (INT3 software breakpoint) and clicking submit on the debugee in order to trigger the breakpoint, x32dbg hangs completely. I can't click anything and if I can't minimize/maximize it. I can tell all input has stopped because my volume keys stop working.

Elaborate reproduction steps for the bug/issue being reported:

• Start byond.exe
• Attach to byond.exe
• Set a breakpoint on DungPager::Login (exported subroutine)
• Go back to debugee and login with an account

[torusrxxx]

Are you sure that the debuggee does not make use of any anti-debugging technique?

[mrexodia]

I also reproduced this in x32dbg when debugging x32dbg that was debugging this target. It appears that the Qt message queue is very full and that it cannot continue but it is really hard to debug...

…

On Fri, 3 Mar 2017 at 11:21, Torusrxxx ***@***.***> wrote: Are you sure that the debuggee does not make use of any anti-debugging technique? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1481 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACWCmRMNVf8sJS4O3UAX5QCCHWCj9lGeks5rh-mjgaJpZM4MRsGr> .

[mrexodia]

@Maktm what plugins do you have? I attempted to reproduce the issue today with x32dbg in debug mode, but I couldn't reproduce it this time. Could you try without plugins and after that also without snowman (use https://github.com/x64dbg/SnowmanDummy/releases)?

Okay, I think the issue is somewhere in snowman. In release mode (with plugins). With snowman is hangs, without snowman it works as expected...

[mrexodia]

Hm no, it turns out it's probably some race condition. I can now reproduce it with a clean release...

[X39]

just stumbled across this bug too today
accidently deleted an older release, debugging my target now freezes with int3 breakpoint (~month maybe two old)

[mrexodia]

I'm having trouble reliably reproducing the issue... Do you know a release that is close to the version that didn't have this issue @X39 ?

[mrexodia]

It appears to be stuck here:

[X39]

sadly no
as said ... removed it accidently and just downloaded the latest

[mrexodia]

Once x32dbg hangs, do the following:

1. Attach to x32dbg.exe with x32dbg
2. Suspend all threads
3. Put a breakpoint on PeekMessageW
4. Replace the code with LoadLibrary("peeker.dll"); call peeker

#include <Windows.h>
#include <stdio.h>

HANDLE hFile;

struct BufferedWriter
{
    explicit BufferedWriter(HANDLE hFile, size_t size = 65536)
        : hFile(hFile),
        mBuffer(new char[size]),
        mSize(size),
        mIndex(0)
    {
        memset(mBuffer, 0, size);
    }

    bool Write(const void* buffer, size_t size)
    {
        return Write((const char*)buffer, size);
    }

    bool Write(const char* buffer, size_t size)
    {
        for(size_t i = 0; i < size; i++)
        {
            mBuffer[mIndex++] = buffer[i];
            if(mIndex == mSize)
            {
                if(!flush())
                    return false;
                mIndex = 0;
            }
        }
        return true;
    }

    ~BufferedWriter()
    {
        flush();
        delete[] mBuffer;
        CloseHandle(hFile);
    }

private:
    HANDLE hFile;
    char* mBuffer;
    size_t mSize;
    size_t mIndex;

    bool flush()
    {
        if(!mIndex)
            return true;
        DWORD written;
        auto result = WriteFile(hFile, mBuffer, DWORD(mIndex), &written, nullptr);
        mIndex = 0;
        return !!result;
    }
};

extern "C" __declspec(dllexport) void peeker()
{
    int count = 0;
    {
        BufferedWriter bufWriter(hFile);
        MSG msg;

        while(PeekMessageW(&msg, 0, 0, 0, PM_REMOVE))
        {
            bufWriter.Write(&msg, sizeof(msg));
            count++;
        }
    }
    char fuck[128] = "";
    sprintf_s(fuck, "%d messages fucked", count);
    MessageBoxA(0, fuck, "fuck!", MB_SYSTEMMODAL);
}

BOOL WINAPI DllMain(
    _In_ HINSTANCE hinstDLL,
    _In_ DWORD     fdwReason,
    _In_ LPVOID    lpvReserved
)
{
    if(fdwReason = DLL_PROCESS_ATTACH)
    {
        char file[MAX_PATH] = "";
        sprintf_s(file, "fuck%d.fuck", rand());
        hFile = CreateFileA(file, GENERIC_WRITE, 0, nullptr, CREATE_ALWAYS, 0, nullptr);
    }
    return TRUE;
}

Now this will empty the message queue correctly, however it will hang on MessageBoxA with the following call stack:

Address  To       From     Size   Comment                                          Party 
0109D5B8 7549E508 7547CDCC 1C     user32.NtUserSetFocus+C                          System
0109D5D4 75478E71 7549E508 2C     user32.MB_DlgProc+111                            System
0109D600 7548F730 75478E71 24     user32._InternalCallWinProc+2B                   System
0109D624 7548F6C8 7548F730 80     user32.InternalCallWinProc+20                    System
0109D6A4 7548F2D7 7548F6C8 5C     user32.UserCallDlgProcCheckWow+129               System
0109D700 7548F4F5 7548F2D7 20     user32.DefDlgProcWorker+A7                       System
0109D720 75478E71 7548F4F5 2C     user32.DefDlgProcW+25                            System
0109D74C 754790D1 75478E71 94     user32._InternalCallWinProc+2B                   System
0109D7E0 754A78E4 754790D1 6C     user32.UserCallWinProcCheckWow+18E               System
0109D84C 7549044B 754A78E4 10C    user32.SendMessageWorker+1DB                     System
0109D958 75494FBA 7549044B 44     user32.InternalCreateDialog+9C6                  System
0109D99C 7549F907 75494FBA CC     user32.InternalDialogBox+F4                      System
0109DA68 7549F06D 7549F907 164    user32.SoftModalMessageBox+E2C                   System
0109DBCC 754E786B 7549F06D 80     user32.MessageBoxWorker+293                      System
0109DC4C 754E77CB 754E786B 34     user32.MessageBoxTimeoutW+6B                     System
0109DC80 754E760B 754E77CB 20     user32.MessageBoxTimeoutA+7B                     System
0109DCA0 754E75D8 754E760B 1C     user32.MessageBoxExA+1B                          System
0109DCBC 1AF811B2 754E75D8 E4     user32.MessageBoxA+18                            User
0109DDA0 6676FA11 1AF811B2 21AF64 peeker.peeker+102                                User
012B8D04 012F6D10 6676FA11 4      qt5core.QEventDispatcherWin32::processEvents+221 User
012B8D08 0109FC3C 012F6D10 3E00C  012F6D10                                         User
012F6D14 012B8D00 0109FC3C 4      0109FC3C                                         User
012F6D18 00000000 012B8D00        012B8D00                                         User

It just deadlocked somewhere in the kernel... But creating a whole new thread that does call MessageBoxA(0, 0, 0, 0); spawns a message box as expected.

Some further research looking at the ReactOS source code: https://doxygen.reactos.org/dc/dd9/focus_8c.html#add28541af7e513708d0b4fc14d43c84e it calls UserEnterExclusive which apparently locks up the GUI. The weirdest thing is that after resuming the threads everything continue to work, except for user input.

[Mattiwatti]

Try the following:

• Windows key + R, type services.msc and hit enter
• Go to 'Windows Font Cache Service'
• Set it to disabled and stop it.

[Mattiwatti]

As a followup, here is the guilty stacktrace:

(this is the x32dbg.exe main thread)

For some reason win32k.sys keeps looking something up in the font cache infinitely. (But it's not stuck doing it, because PeekMessage does return.) However what the actual underlying cause is I don't know. win32k.sys is well known for being a buggy piece of shit, but there could also be something else at play.

I actually had some trouble reproducing the stacktrace after disabling the font cache, because even after re-enabling it I could still set the breakpoint and step. I had to reboot to get it to hang again.

Edit: I made a new stacktrace after fixing my symbols, because the offsets were fishy. So now it looks like it doesn't have anything to do with the font cache at all - even though disabling it did fix the problem for me. Huh?!

[mrexodia]

Yeah, this issue is really hard to grasp. First I could reproduce it every single time, then I tried compiling x64dbg in debug mode, couldn't reproduce it. Then I tried release, reproduced it again, then I removed snowman from release and it didn't happen for pretty long, but then it happened again. Now I have to kind of keep trying to reproduce it and it happens occasionally. Something I did observe though, sometimes it hangs with 15% (I have 6 cores) CPU usage and sometimes it hangs with the usual 5-6%...

[Maktm]

what plugins do you have?

Just the defaults that come with x32dbg/x64dbg. I was using a fresh install when this issue came up.

I attempted to reproduce the issue today with x32dbg in debug mode, but I couldn't reproduce it this time.

The issue only happens sometimes so it's pretty annoying to debug.